{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4e70972a",
   "metadata": {},
   "source": [
    "<table align=\"left\">\n",
    "  <td>\n",
    "    <a href=\"https://colab.research.google.com/github/nyandwi/machine_learning_complete/blob/main/7_intro_to_artificial_neural_networks_and_tensorflow/3_neural_networks_for_regresion_with_tensorflow.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78a89ab1",
   "metadata": {},
   "source": [
    "*This notebook was created by [Jean de Dieu Nyandwi](https://twitter.com/jeande_d) for the love of machine learning community. For any feedback, errors or suggestion, he can be reached on email (johnjw7084 at gmail dot com), [Twitter](https://twitter.com/jeande_d), or [LinkedIn](https://linkedin.com/in/nyandwi).*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "304ad719",
   "metadata": {
    "id": "304ad719"
   },
   "source": [
    "<a name='0'></a>\n",
    "# Neural Networks for Regression with TensorFlow"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6a095ea",
   "metadata": {
    "id": "b6a095ea"
   },
   "source": [
    "### Contents \n",
    "\n",
    "* [1. Intro to Regression with TensorFlow](#1)\n",
    "* [2. Starting Simple: Fitting a Straight Line](#2)\n",
    "   * [2.1 Gathering the data](#2-1)\n",
    "   * [2.2 Looking in the data](#2-2)\n",
    "   * [2.3 Preparing the data for a model](#2-3)\n",
    "   * [2.4 Creating, Compiling and Training a model](#2-4)\n",
    "   * [2.5 Evaluating a model](#2-5)\n",
    "   * [2.6 Improving a model](#2-6)\n",
    "  \n",
    "\n",
    "* [3. Going Beyond: A Real world dataset](#3)\n",
    "   * [3.1 Gathering the data](#3-1)\n",
    "   * [3.2 Looking in the data](#3-2)\n",
    "   * [3.3 Preparing the data for a model](#3-3)\n",
    "   * [3.4 Creating, Compiling and Training a model](#3-4)\n",
    "   * [3.5 Evaluating a model](#3-5)\n",
    "   * [3.6 Improving a model](#3-6)\n",
    "   * [3.7 Saving a model](#3-7)\n",
    "   * [3.8 Final notes](#3-8)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "AAqeb2v2jSWT",
   "metadata": {
    "id": "AAqeb2v2jSWT"
   },
   "source": [
    "<a name='1'></a>\n",
    "## Intro to Regression with TensorFlow"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hcDOlUdZxDGu",
   "metadata": {
    "id": "hcDOlUdZxDGu"
   },
   "source": [
    "We know neural networks in taking moonshots but they can also be used for regression problems. \n",
    "\n",
    "In regression task/problems, we are interested in predicting the a single or multiple continous numbers. \n",
    "\n",
    "Take an example of house price prediction. We can be given the properties/features of the house such as size, region, and number of bedrooms to predict the price of such house. This example is a single number prediction, also termed as `univariate regression`. \n",
    "\n",
    "Another example appears in object detection(recognizing & localizing image). In order to localize the object with the bounding box, you have got to find the coordinates of the object's center and the bounding box. The prediction of these coordinates is an example of predicting multiple values at once. It is usually termed as `multivariate regression`. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "jG7bLsj03uxw",
   "metadata": {
    "id": "jG7bLsj03uxw"
   },
   "source": [
    "Any typical architecture for a regression neural network will have common values or ranges of values of `hyperparameters` (hyperparameters are all parameters that you as engineer has to set, such as learning rate, number of layers, etc..). \n",
    "Let's discuss them. \n",
    "\n",
    "\n",
    "#### **Input, hidden, and Output Layers**\n",
    "\n",
    "The input layer usually has the neurons(or units) equivalent to the number of input features. For example, if our house dataset has 10 variables, 9 input training features, 1 target feature(price of house), then the input neurons will be 9. \n",
    "\n",
    "The number of hidden layers depend on the problem and the size of the dataset, but generally, it will be between 1 and 5. Same is true about the number of neurons in each hidden layer, it will depend on the problem & dataset size, but generally, neurons can be between 10 to 100. \n",
    "\n",
    "The number of neurons in output layer will depend on the problem. If you are predicting a single number, it will be 1. If you are predicting the coordinates of the object's center and bounding box during object detection, it will be 4 (because there are 4 coordinates, 2 for object's center, 2 for height/width of the box). \n",
    "\n",
    "\n",
    "#### **Activation Function**\n",
    "\n",
    "The choice of activation function depends on the problem, but in most cases, `relu` will work well in hidden layers. \n",
    "\n",
    "The activation function in the output layer is very specific on the goal of the problem. Unlike neural network classifiers that usually use [`sigmoid`](https://en.wikipedia.org/wiki/Sigmoid_function) or [`softmax`](https://en.wikipedia.org/wiki/Softmax_function), regressor doesn't need to have an activation since you want the output values as they are. That being said, you may want to prevent the negative values in the output layer. In that case, you can use [`ReLU`](https://en.wikipedia.org/wiki/Rectifier_(neural_networks)). \n",
    "\n",
    "\n",
    "#### Training Loss Function\n",
    "\n",
    "The loss function used in regression is usually [`Mean Squared Error(MSE)`](https://en.wikipedia.org/wiki/Mean_squared_error). When you are aware that your dataset contain outliers, you can use [`Mean Absolute Error (MAE)`](https://en.wikipedia.org/wiki/Mean_absolute_error). MAE can potentialy be used in time series prediction since that type of data tends to have outliers.\n",
    "\n",
    "Another loss function that is used alot if [Huber loss](https://www.tensorflow.org/api_docs/python/tf/keras/losses/huber). It is combination of both MSE and MAE. \n",
    "\n",
    "#### Optimizer\n",
    "\n",
    "A good rule of thumb when choosing an optimizer is to start with [Adam](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/Adam). There are other optimizers that you can try, such as SGD(Stockastic Gradient Descent), RMSProp, Nadam, etc...Learn more about them on [TensorFlow optimizer documentation page](https://www.tensorflow.org/api_docs/python/tf/optimizers). \n",
    "\n",
    "Below is a summary of hyerparameter best practices in neural network regressors.\n",
    "\n",
    "\n",
    "\n",
    "| **Hyperparameter** | **Typical value** |\n",
    "| --- | --- |\n",
    "| Neurons at input layer | 1 neuron per feature |\n",
    "| No of hidden layer(s) | depend on problem, start from 1 to 10 |\n",
    "| Neurons per hidden layer | depend on problem, generally 10 to 100 |\n",
    "| Neurons at input layer | depend on the desired result, 1 for univariate regression |\n",
    "| Activation in hidden layers| Relu or its variants(LeakyReLU, SeLU|\n",
    "| Activation in output layer |None in most cases|\n",
    "| Loss function | MSE or MAE |\n",
    "| Optimizer | SGD, Adam, RMSProp |\n",
    "\n",
    "*Table: Typical values of hyperparameters in neural network regressors*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8H7r9hKeHAeM",
   "metadata": {
    "id": "8H7r9hKeHAeM"
   },
   "source": [
    "There are many hyperparameters in neural networks and finding the best values of each and each can be overwhelming. \n",
    "\n",
    "When choosing hyperparameters, it is advised to use hyperparameter tuning tools such as [Keras Tuner](https://keras.io/keras_tuner/) to search the best hyperparameters whenever possible. It is nearly impossible to assume that a given value of hyperparamater will work well at first. We usually have to experiment with different values, and a good tool can help you do that quickly. \n",
    "\n",
    "Let's put all of the above into practice, starting simple, and later, taking a futher step into real world dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9pEQ8VbkIRsO",
   "metadata": {
    "id": "9pEQ8VbkIRsO"
   },
   "source": [
    "<a name='2'></a>\n",
    "\n",
    "## 2. Starting Simple: Fitting a Straight Line"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "I8spI9b6g-vy",
   "metadata": {
    "id": "I8spI9b6g-vy"
   },
   "source": [
    "One of the simplest things we can model is perhaps a linear equation(it has been proven that [neural networks can model any mathematical function](http://neuralnetworksanddeeplearning.com/chap4.html)). \n",
    "\n",
    "So, a linear equation forms a straight line. Its form is `y=aX+b` where a is coefficient (or `weight`) and b is intercept (or `bias`). \n",
    "\n",
    "Let's assume that we have this equation `y=2X+1` and we are interested in using neural networks to predict y given any value of X. \n",
    "\n",
    "But we will start with creating our data based off such equation. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "AhILzCKOg9mu",
   "metadata": {
    "id": "AhILzCKOg9mu"
   },
   "source": [
    "<a name='2-1'></a>\n",
    "### 2.1  Gathering the data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "N5UGgGiOk7xO",
   "metadata": {
    "id": "N5UGgGiOk7xO"
   },
   "source": [
    "Let's first import the libraries. I will import `TensorFlow`, `NumPy` and `Matplolib` for plotting the straight line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "jhhUN8KYlHCd",
   "metadata": {
    "id": "jhhUN8KYlHCd"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow import keras \n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98AuRTg8lSPd",
   "metadata": {
    "id": "98AuRTg8lSPd"
   },
   "source": [
    "After we have imported all relevant libraries, it's time to create our data. We only have one input feature (`X`) and output label `y`. \n",
    "\n",
    "We can either create it with `tf.constant()` or `np.array()`. Both will work, but for convenience, let's use `tf.constant()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7f6p8Csdl8Vc",
   "metadata": {
    "id": "7f6p8Csdl8Vc"
   },
   "outputs": [],
   "source": [
    "# Create input feature X\n",
    "\n",
    "X = tf.constant([-2,-1,0.0,1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,10,12])\n",
    "\n",
    "# Create label y\n",
    "\n",
    "y = tf.constant([-3.0,-1.0,1.0,3.0,5.0,7.0,9.0,11.0,13.0,15.0,17.0,21.0,25.0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "nwPHdU4ZqUbK",
   "metadata": {
    "id": "nwPHdU4ZqUbK"
   },
   "source": [
    "<a name='2-2'></a>\n",
    "### 2.2  Looking in the Data\n",
    "\n",
    "It's always good to look in the data. This can be done in many ways and it depend on the kind of the dataset. If you're working with images, you might want to go through some images and their labels looking if there are no mislabelling. \n",
    "\n",
    "In structured data, or data in tabular form, you might have to visualize the distribution of individual features or plot their relationship. \n",
    "\n",
    "For us now, we have a simple data. Let's scatter plot X and y."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "YKQFuvyLqkA6",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 296
    },
    "id": "YKQFuvyLqkA6",
    "outputId": "cdeb8cbe-8633-4bc1-c296-ebc04194df1c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'y')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3hUdd7+8fcnIYFQQ+8QQKp0AyzYUFRQWGlbdFUUVrHtY/ntgqDY1obiWp7d1RXFruujNAsqIqjYFQSTUCItlCQkoSQESCAk398fGRECgYCZOVPu13VxkTlnkrmDeOfwPed8xpxziIhI5IjyOoCIiASWil9EJMKo+EVEIoyKX0Qkwqj4RUQiTBWvA1REgwYNXEJCgtcxRERCytKlS7c55xqW3R4SxZ+QkMCSJUu8jiEiElLMbOPRtmupR0Qkwqj4RUQijIpfRCTCqPhFRCKMil9EJML4rfjNrKWZfWJmK81shZnd7Nt+j5mlm9ly36+L/JVBRCRUzV2WzulTF9Fm0jxOn7qIucvSK+1r+/NyzgPAX51zP5hZLWCpmS3w7XvcOfeoH19bRCRkzV2WzuTZyRQUFQOQnlvA5NnJAIzo1fxXf32/HfE75zKdcz/4Ps4HVgG/PrGISJibNj/1YOn/rKComGnzUyvl6wdkjd/MEoBewLe+TX8xsyQze97M6pbzOePNbImZLcnJyQlETBERzznnSM8tOOq+jHK2nyi/F7+Z1QRmAbc453YBTwPtgJ5AJvCPo32ec266cy7ROZfYsOERdxyLiISd7F2FXPvK0nL3N4uPq5TX8Wvxm1kMpaX/mnNuNoBzLss5V+ycKwGeBfr6M4OISLBzzvHm95sZ9NhnfPZTDhf3aEa1KofXc1xMNBMGd6yU1/PbyV0zM2AGsMo599gh25s65zJ9D0cCKf7KICIS7DZt38vkOUl8uXY7fdvU4+HR3WnToAZzl6UzbX4qGbkFNIuPY8LgjpVyYhf8e1XP6cAVQLKZLfdtux241Mx6Ag5IA671YwYRkaBUXOJ48as0Hp2fSnSUcf+IrvypbyuiogwovXqnsoq+LL8Vv3PuC8COsut9f72miEgoWJOVz8RZSSzblMs5HRvywMhulbZ+XxEhMZZZRCQc7D9Qwn8+W8e/Fq2lRtVonvhjT4b3bEbpynjgqPhFRAIgaUsuE2cmsXprPr/t0Yy7f9uFBjWrepJFxS8i4keFRcU8vuAnnv18PQ1rVeXZMYmc36Wxp5lU/CIifvLN+u1MmpVE2va9XNq3JZMv6kztajFex1Lxi4hUtvzCIqZ+sJrXvt1Eq3rVef3qfgw4pYHXsQ5S8YuIVKJFq7O4Y04KWbsKufqMNvz1go7ExUZ7HeswKn4RkUqwY89+/v7uCuYuz6BD45o8ddkAerU66igyz6n4RUR+Becc7yZlcs87K8gvLOLmQe258ZxTiK0SvO9zpeIXETlJW/MKmTI3hY9XZdGjRR0e/l0/OjWp7XWs41Lxi4icIOccb3y/mQfnraKopIQpQzsz9vQ2REcF9kask6XiFxE5ARu372HSrGS+Xr+d/m3rM3V0N1rXr+F1rBOi4hcRqYDiEscLX27g0Y9SiYmK4qFR3bikT8uAj1uoDCp+EZHjSN1aOlTtx825nNe5EfeP6EaTOtW8jnXSVPwiIuXYf6CEpz5dy78/WUutajH876W9+G33piF5lH8oFb+IyFEs35zLbTOTSM3KZ0TPZtz121OpVyPW61iVQsUvInKIgv3F/OOjVJ7/cgONa1fj+asSObeTt0PVKpuKX0TE56t125g0K5lNO/ZyWb9WTLqwE7WCYKhaZVPxi0jE21VYxEPvr+K/320moX513hj/G37Ttr7XsfxGxS8iEe3jlVncMTeZnPx9XHtWW245r0PQDVWrbCp+EYkIc5elM21+Khm5BTSLj+P6s9vxbdoO3v0xg05NavHsmES6t4j3OmZAqPhFJOzNXZbO5NnJFBQVA5CeW8CUt1OIjjL+3/kduO7sdkE9VK2yqfhFJOxNm596sPQPVb9GLDcNau9BIm9Fzo84EYlY6bkFR92ek78vwEmCg4pfRMLahm17iI0+etU1i48LcJrgoKUeEQlLB4pLmPHFBh5b8BNRURCDUVTsDu6Pi4lmwuCOHib0jopfRMLOqsxd3DYriaQteZzfpTH3j+jK1+u2H3ZVz4TBHRnRq7nXUT3ht+I3s5bAy0BjwAHTnXNPmlk94P+ABCAN+INzbqe/cohI5Nh3oJh/L1rLU5+uI756DP/+U28u6tYEM2NEr+YRW/Rl+fOI/wDwV+fcD2ZWC1hqZguAq4CFzrmpZjYJmATc5sccIhIBlm7cyW2zklibvZtRvZtz59Au1A2ToWqVzW/F75zLBDJ9H+eb2SqgOTAcGOh72kvAp6j4ReQk7d1/gGnzU3nxqzSa1q7GC2P7cE7HRl7HCmoBWeM3swSgF/At0Nj3QwFgK6VLQSIiJ+yLNduYNDuJLTsLGNO/NROHdKJmVZ26PB6//wmZWU1gFnCLc27XoW9g4JxzZubK+bzxwHiAVq1a+TumiISQvIIiHpi3kjeXbKFNgxq8eW1/+rap53WskOHX4jezGEpL/zXn3Gzf5iwza+qcyzSzpkD20T7XOTcdmA6QmJh41B8OIhJ55q/Yyp1zU9i+Zz/XD2zHzYPaUy0mvIeqVTZ/XtVjwAxglXPusUN2vQNcCUz1/f62vzKISPjIyd/HPe+sYF5yJl2a1ub5q/rQtXkdr2OFJH8e8Z8OXAEkm9ly37bbKS38N83sz8BG4A9+zCAiIc45x5xl6fz9vZXs3VfMhMEdGX9WW2LKuRtXjs+fV/V8AZT3jsSD/PW6IhI+0nMLuH12Mp/9lMNprevy8OjunNKoptexQp5Of4tI0Ckpcbz67UYe/mA1Drjnt10Y0z+BqKjyjiXlRKj4RSSorMvZzaRZSXyftpMz2zfgwZHdaFmvutexwoqKX0SCwoHiEqZ/vp4nPl5DXEw0j/6+B6N7N+fQS8Clcqj4RcRzKzLyuG1WEinpu7iwaxPuHX4qjWpV8zpW2FLxi4hnCouK+eeiNfzns/XUrR7L05f15sJuTb2OFfZU/CLiiSVpO5g4K4n1OXv43WktmDK0M/HVNVQtEFT8IhJQe/aVDlV76es0mtWJ4+VxfTmrQ0OvY0UUFb+IBMzin3KYPDuZjLwCruyfwITBHamhoWoBpz9xEfG73L37uX/eKmYu3UK7hjV469r+JCZoqJpXVPwi4lcfJGdy59sr2Ll3P3855xT+cu4pGqrmMRW/iPhF9q5C7np7BR+u2MqpzWrz0rg+nNpMQ9WCgYpfRCqVc46ZS7dw33srKTxQwm1DOnHNmW2ooqFqQUPFLyKVZvOOvdw+J5nP12yjT0Jdpo7uTruGGqoWbFT8InLS5i5LZ9r8VNJzC6gTF0NBUTExUcZ9w0/lsn6tNVQtSKn4ReSkzF2WzuTZyRQUFQOlb4cYZTB5SBeu6J/gbTg5Ji26ichJeeTD1QdL/2clDp77YoNHiaSiVPwicsJS0vPIyCs86r6M3IIAp5ETpaUeEamwwqJinvh4Dc9+vp4oKz3CL6tZfFzgg8kJUfGLSIV8t2EHk2YlsX7bHv6Y2JIeLetw33urDlvuiYuJZsLgjh6mlIpQ8YvIMeUXFvHIh6m88s1GWtaL47Wr+3H6KQ0AqB5bhWnzU8nILaBZfBwTBndkRK/mHieW41Hxi0i5PknN5o7ZyWTuKmTc6W342+AOVI/9pTZG9Gquog9BKn4ROcLOPfu5772VzF6WTvtGNZl1/QB6t6rrdSypJCp+ETnIOce85EzufnsFeQVF3DSoPTee046qVTRULZyo+EUEgKxdhUyZm8KClVl0b1GHV6/uR+emtb2OJX6g4heJcM453lyymfvnrWL/gRJuv6gT407XULVwpuIXiWCbtu9l0uwkvlq3nX5t6vHw6O4kNKjhdSzxMxW/SAQqLnG88OUGHv0olSpRUTw4shuX9GmpoWoRwm/Fb2bPA8OAbOdcV9+2e4BrgBzf0253zr3vrwwiUurnKZoZuQU0rFWVuJhoNu7Yy7mdGvHAyK40raO7bSOJP4/4XwT+BbxcZvvjzrlH/fi6InKIslM0s/P3AXDFb1rz9+GnYqaj/Ejjt7M3zrnFwA5/fX0RqZhp81OPmKIJsGh1tko/Qnlx2v4vZpZkZs+bWbl3hJjZeDNbYmZLcnJyynuaiBxDwf5i0suZlqkpmpEr0MX/NNAO6AlkAv8o74nOuenOuUTnXGLDhg0DlU8kbHy9bjtDnlxc7n5N0YxcAS1+51yWc67YOVcCPAv0DeTri0SCXYVFTJ6dzKXPfgPAjQPbERdz+J23mqIZ2QJ6OaeZNXXOZfoejgRSAvn6IuFu4aos7piTQnZ+IePPasut53UgLjaa9o1raYqmHOTPyzn/CwwEGpjZFuBuYKCZ9QQckAZc66/XF4kk23fv4953V/LOjxl0bFyL/1xxGj1bxh/crymacii/Fb9z7tKjbJ7hr9cTiUTOOd75MYN7311JfmERt57XgesHtiO2isYtSPl0565IiMrMK2DKnBQWrs6mR8t4HhndnY5NankdS0KAil8kxJSUON74fjMPvb+KopISpgztzNjT2xCtcQtSQSp+kRCStm0Pk2Yn8c36HQxoV5+HRnWjdX0NVZMTo+IXCQHFJY7nv9jAPxakEhMVxdRR3fhjn5a681ZOiopfJMilbs1n4swf+XFLHud1bsz9I7rSpE41r2NJCFPxiwSRQ6doNq1TjW4t6rBodTa1q8Xwz0t7Max7Ux3ly6+m4hcJEmWnaGbkFZKRV0hi67pMH5NIvRqxHieUcKGLfUWCRHlTNDPzClX6UqlU/CJBQlM0JVC01CPisbyCIh56f1W5+zVFUyqbil/EQwtWZjFlbjI5+fs4t1Mjvlq7jcIDJQf3a4qm+IOKX8QD23bv4553VvBeUiadmtTi2TGJdG8Rf9hVPZqiKf6i4hcJIOccby/P4J53V7B3XzF/Pb8D1w1sR0x06ek2TdGUQFDxiwRIRm4BU+amsGh1Nr1alQ5Va99YQ9Uk8FT8In5WUuJ47btNPPzBaopLHHcN68KVAxI0VE08o+IX8aP1ObuZNCuZ79J2cMYpDXhoVDda1qvudSyJcCp+ET84UFzCc19s4PEFPxFbJYpHRnfn94ktNG5BgoKKX6SSrczYxcRZP5KSvosLujTmvhFdaVxbQ9UkeKj4RSrJvgPF/GvRWp7+dB3x1WN46rLeXNi1iY7yJeio+EVOQtnr7X93WgvmJWeyNns3o3o3586hXair+ToSpFT8Iieo7BTN9NwCnly4hrrVY3hxbB8GdmzkcUKRYzvukDYz+x8zqxuIMCKhoLwpmtViolX6EhIqMp2zMfC9mb1pZkNMC5YS4cqbork1rzDASUROznGL3zk3BWgPzACuAtaY2YNm1s7P2USCzocpmZR335WmaEqoqNA8fuecA7b6fh0A6gIzzewRP2YTCRrZ+YXc8NpSrnv1B5rWiaNqlcP/19EUTQklxz25a2Y3A2OAbcBzwATnXJGZRQFrgIn+jSjiHeccs35I5773VlJQVMyEwR0Zf1Zb5iVlaoqmhKyKXNVTDxjlnNt46EbnXImZDSvvk8zseWAYkO2c6+rbVg/4PyABSAP+4JzbeXLRRfxry8693D4nhcU/5XBa67o8PLo7pzSqCWiKpoS2iqzx31229A/ZV/7bBsGLwJAy2yYBC51z7YGFvsciQaWkxPHSV2lc8PhilqTt4N6LT+Wta/sfLH2RUOe36/idc4vNLKHM5uHAQN/HLwGfArf5K4PIiVqXs5vbZiaxZONOzmzfgAdHaqiahJ9A38DV2DmX6ft4K6WXih6VmY0HxgO0atUqANEkkhUVlzB98XqeXLiGuJhoHv19D0b3bq5xCxKWPLtz1znnzMwdY/90YDpAYmJiuc8T+bVS0vO4bVYSKzJ2cVG3Jtxz8ak0qqWhahK+Al38WWbW1DmXaWZNgewAv77IQYVFxfzvwjU8s3g9davH8p/LezOka1OvY4n4XaCL/x3gSmCq7/e3A/z6IgB8n7aD22YmsX7bHn5/WgumDO1CneoxXscSCQi/Fb+Z/ZfSE7kNzGwLcDelhf+mmf0Z2Aj8wV+vL/KzQydpNqlTjXYNa/LF2m00j4/j5XF9OatDQ68jigSUP6/qubScXYP89ZoiZZWdpJmZV0hmXiFntW/A05efRo2qGlArkadCIxtEQlV5kzTX5exR6UvEUvFLWCtvkmZGOdtFIoGKX8JS9q5Crn1lSbn7NUlTIpmKX8KKc443l2zmvMc+45PUHIZ1b0o1TdIUOYwWOSVsbN6xl8mzk/li7Tb6JtRj6uhutG1Y84j3x9UkTYl0Kn4JecUljpe/TuORD1OJMrhv+Klc1q81Ub53TNEkTZHDqfglpK3NzmfizCR+2JTLwI4NeWBkN5pr/V7kmFT8EpKKikt45rN1/O/CtVSvGs3jf+zBiJ4aqiZSESp+CTnJW/KYMPNHVm/NZ2j3ptx78ak0qFnV61giIUPFLyGjsKiYxz/+iWcXr6dBzao8c8VpDD61idexREKOil9CwrfrtzNpdjIbtu3hj4ktuX1oZ+rEaaiayMlQ8UtQyy8s4uEPV/PqN5toWS+O167ux+mnNPA6lkhIU/FL0Ch7vf3Qbk15LymDzF2F/PmMNvz1gg5Uj9VfWZFfS/8XSVAoO0UzPbeA6Z+vp0ntasy6fgC9W9X1OKFI+NDIBgkK5U3RjDJU+iKVTMUvQaG8KZqZeYUBTiIS/rTUI55yzvF/32/GAHeU/ZqiKVL5VPzimU3b9zJpdhJfrdtOu4Y12LKzgH0HSg7u1xRNEf9Q8UvAFZc4XvhyA49+lEqVqCgeGNmVS/u04p0fMzRFUyQAVPwSUD9llQ5VW745l3M7NeKBkV1pWqd0OUdTNEUCQ8UvAbH/QAlPf7qOf32yhlrVYnjykp5c3KOZhqqJeEDFL3734+ZcJs5MIjUrn+E9m3HXsC7U11A1Ec+o+MVvCvYX89iCVGZ8sYFGtarx3JhEzuvS2OtYIhFPxS9+8fW67UyancTG7Xv5U79WTLqwE7WraaiaSDBQ8Uul2lVYxEPvr+a/322idf3qvH5NPwa001A1kWCi4pdKs3BVFnfMSSE7v5DxZ7Xl1vM6EBcb7XUsESnDk+I3szQgHygGDjjnEr3IISen7BTN689ux3dpO3jnxww6NanFM1ecRo+W8V7HFJFyeHnEf45zbpuHry8n4WhTNKe8nUJ0lHHreR24fmA7YqtoBJRIMNNSj5yQ8qZo1q8Ry83ntfcgkYicKK8OzRzwkZktNbPxR3uCmY03syVmtiQnJyfA8aQ85U3RzMnfF+AkR7rsssvo2LEjXbt2Zdy4cRQVFQHw2GOPMW7cuIPPe+211xg6dGiFv+6QIUOIj49n2LBhlZ5ZxAteFf8ZzrnewIXAjWZ2VtknOOemO+cSnXOJDRs2DHxCOULatj3lLuMEwxTNyy67jNWrV5OcnExBQQHPPfccADfddBM//PADX375Jbm5uUyZMoV//vOfR3z+wIEDSUtLO2L7hAkTeOWVV/wdXyRgPCl+51y67/dsYA7Q14scUjEHikuYvngdg59YTJRBTPThYxYqe4rmXXfdxRNPPHHw8R133MGTTz553M+76KKLMDPMjL59+7JlyxYAqlSpwlNPPcWNN97IxIkTGTduHG3btq1wnkGDBlGrVq0T/0ZEglTA1/jNrAYQ5ZzL9318AfD3QOeQilm9dRe3zUzixy15nN+lMfeP6MrX67b7dYrmuHHjGDVqFLfccgslJSW88cYbLFq0iJ49ex71+a+//jpdunQ5+LioqIhXXnnlsB8WAwYMoHPnznz88cesWrWq0rKKhCIvTu42Bub4hnNVAV53zn3oQQ45hn0Hivn3J+t46pO11ImL4V9/6sXQbk0xM79P0UxISKB+/fosW7aMrKwsevXqRevWrVm+fHmFPv+GG27grLPO4swzzzy4bffu3SxZsoSioiJycnJo0aIFAC+88MLBHxBr167loosuIjY2ljZt2jBnzpzK/+ZEgkDAi985tx7oEejXlYr7YdNObpuZxJrs3Yzs1Zy7hnWhbo3YgGa4+uqrefHFF9m6dSvjxo0jPz//sCI/1KFH/Pfeey85OTk888wzhz3n7rvv5vLLL6dx48bceuutvPXWWwCMHTuWsWPHAqVr/C+++CIJCQn++8ZEgoAu55SD9u4/wD8++onnv9xAk9rVeOGqPpzTqZEnWUaOHMldd91FUVERr7/+OtHR0cc94n/uueeYP38+CxcuJCrql9NXycnJzJs3j+XLlxMbG8uMGTNYsGAB559/vr+/DZGgpOIXAL5cu41Js5PYvKOAK37TmolDOlLLw6FqsbGxnHPOOcTHxxMdXbGxD9dddx2tW7emf//+AIwaNYo777yT66+/nscff5xq1aoB8PTTTzNmzJiDPwiO58wzz2T16tXs3r2bFi1aMGPGDAYPHnzy35yIx8y5o73FdXBJTEx0S5Ys8TpGWMorKOKh91fxxvebadOgBlNHdaNf2/pex6KkpITevXvz1ltv0b69bgwTORlmtvRoI3F0xB/BPlqxlSlzU9i+Zz/Xnd2OW85rT7UY74eqrVy5kmHDhjFy5EiVvogfqPgjUE7+Pu55dwXzkjLp3LQ2M67sQ7cWdbyOdVCXLl1Yv3691zFEwpaKP4I455i7PJ17313J3n3F/O2CDlx7djtiojVUTSSSqPjD2KHjkxvVrkq9GrGsysynd6t4Hvldd05ppLtRRSKRij9MlR2fnLVrH1m79jGyV3Me/X0PoqPsOF9BRMKV/o0fpsobn/zdhh0qfZEIp+IPQweKS8odn5xRznYRiRwq/jCzMmMXI576stz9wTA+WUS8peIPE4VFxTw6P5WL//UFW/P2MXZAAnFlrsmv7PHJIhKadHI3DCzduIOJM5NYl7OH0b1bcOewzsRXj6VHy3i/jk8WkdCk4g9he/YdYNr8VF76Oo1mdeJ4aVxfzu7wy7uV+Xt8soiEJhV/iPp8TQ6TZyeTnlvAmN+0ZsKQTtSsqv+cInJ8aooQk7e3iPvnreStpVto27AGb17bnz4J9byOJSIhRMUfQj5MyeTOt1ewY89+bhjYjpsGBcdQNREJLSr+EJCdX8jdb6/gg5StdGlamxeu6kPX5sEzVE1EQouKP4g555j1Qzr3vbeSgqJiJg7pyDVnttVQNRH5VVT8QWrLzr3cPieFxT/l0CehLlNHd6ddw5pexxKRMKDiDxI/T9JMzy2gTlwMBUXFxEQZfx9+Kpf3a02U5uuISCVR8QeBspM08wqKiDKYNKQLY/oneBtORMKOFouDwCMfrj5ikmaJgxlfbPAokYiEMxW/x1LS88jIKzzqPk3SFBF/0FKPRwqLinly4RqmL15PlJUe4ZelSZoi4g8qfg98n7aD22YmsX7bHv6Q2IKeLeO5771Vhy33aJKmiPiLij+Adu87wCMfrublrzfSom4cr/65H2e0bwBA9dgqmqQpIgHhSfGb2RDgSSAaeM45N9WLHIH0aWo2d8xJISOvgLGnJ/C3CzpS45ChapqkKSKBEvDiN7No4N/A+cAW4Hsze8c5tzLQWQJh55793DdvJbN/SOeURjWZed0ATmtd1+tYIhLBvDji7wusdc6tBzCzN4DhQFgVv3OOD1K2ctfbKeTuLeJ/zj2Fv5x7ClWraKiaiHjLi+JvDmw+5PEWoF/ZJ5nZeGA8QKtWrQKTrJJk7yrkzrdTmL8ii27N6/DyuH50aVbb61giIkAQn9x1zk0HpgMkJiYe5WLH4OOc460lW7h/3kr2HShh8oWd+PMZbaiioWoiEkS8KP50oOUhj1v4toW0zTv2Mnl2Ml+s3UbfNvWYOqobbTVUTUSCkBfF/z3Q3szaUFr4lwB/8iBHpSgucbz0VRrT5qcSHWXcP6Irf+rbSkPVRCRoBbz4nXMHzOwvwHxKL+d83jm3ItA5TtbPUzQzcgtoWKsq1WOjSdu+l4EdG/LgyG6621ZEgp4na/zOufeB97147V+j7BTN7Px9AFzerxX3jeiKmY7yRST46azjCZg2P/WIKZoAn6TmqPRFJGSo+CuosKiY9HKmZWqKpoiEEhV/BXyzfjtDnlhc7n6t64tIKFHxH0N+YRF3zEnmkunfUOLghoHtiIs5/M5bTdEUkVATtDdwee2T1dncPieZrF2FXH1GG/7fBR2oHluFDo1raYqmiIQ0FX8ZO/bs5+/vrmDu8gzaN6rJU9cPoFerX4aqaYqmiIQ6Fb+Pc453kzK5550V7Coo4uZB7bnhnHYaqiYiYUfFD2zNK2TK3BQ+XpVF9xZ1eOSafnRqoqFqIhKeIrr4nXO88f1mHpy3iv3FJdxxUWfGnp6goWoiEtYitvg3bt/DpFnJfL1+O/3a1OPh0d1JaFDD61giIn4XccVfXOJ44csNPPpRKlWionhwZDcu6dNSQ9VEJGJEVPGnbs1n4qwkftycy6BOjbh/ZFea1tHNVyISWcK2+A+dotm0TjW6t4hn4eosalWL4clLenJxj2aaryMiESksi7/sFM2MvEIy8rZyWqu6TB9zGvVrVvU4oYiId8Ly8pXypmhu3VWo0heRiBeWxV/etExN0RQRCdPiL29apqZoioiEafFPGNxRUzRFRMoRlid3fx6ipimaIiJHCsviB03RFBEpT1gu9YiISPlU/CIiEUbFLyISYVT8IiIRRsUvIhJhzDnndYbjMrMcYONJfnoDYFslxvG3UMobSlkhtPKGUlYIrbyhlBV+Xd7WzrmGZTeGRPH/Gma2xDmX6HWOigqlvKGUFUIrbyhlhdDKG0pZwT95tdQjIhJhVPwiIhEmEop/utcBTlAo5Q2lrBBaeUMpK4RW3lDKCn7IG/Zr/CIicrhIOOIXEZFDqPhFRCJMRBS/mU0zs9VmlmRmc8ws3utMZZnZEDNLNbO1ZjbJ6zzHYmYtzewTM1tpZivM7GavMx2PmUWb2TIze8/rLMdjZvFmNtP3d3aVmfX3OlN5zOxW39+BFDP7r5lV8zrTodhhUd8AAAQZSURBVMzseTPLNrOUQ7bVM7MFZrbG93tdLzP+rJysfumuiCh+YAHQ1TnXHfgJmOxxnsOYWTTwb+BCoAtwqZl18TbVMR0A/uqc6wL8BrgxyPMC3Ays8jpEBT0JfOic6wT0IEhzm1lz4CYg0TnXFYgGLvE21RFeBIaU2TYJWOicaw8s9D0OBi9yZFa/dFdEFL9z7iPn3AHfw2+AFl7mOYq+wFrn3Hrn3H7gDWC4x5nK5ZzLdM794Ps4n9JiCto3PzCzFsBQ4DmvsxyPmdUBzgJmADjn9jvncr1NdUxVgDgzqwJUBzI8znMY59xiYEeZzcOBl3wfvwSMCGiochwtq7+6KyKKv4xxwAdehyijObD5kMdbCOIiPZSZJQC9gG+9TXJMTwATgRKvg1RAGyAHeMG3NPWcmdXwOtTROOfSgUeBTUAmkOec+8jbVBXS2DmX6ft4K9DYyzAnoNK6K2yK38w+9q0zlv01/JDn3EHpMsVr3iUNH2ZWE5gF3OKc2+V1nqMxs2FAtnNuqddZKqgK0Bt42jnXC9hD8CxFHMa3Nj6c0h9WzYAaZna5t6lOjCu9nj3or2mv7O4Km7dedM6dd6z9ZnYVMAwY5ILv5oV0oOUhj1v4tgUtM4uhtPRfc87N9jrPMZwOXGxmFwHVgNpm9qpzLlgLaguwxTn387+gZhKkxQ+cB2xwzuUAmNlsYADwqqepji/LzJo65zLNrCmQ7XWgY/FHd4XNEf+xmNkQSv+pf7Fzbq/XeY7ie6C9mbUxs1hKT5C943GmcpmZUboGvco595jXeY7FOTfZOdfCOZdA6Z/roiAufZxzW4HNZtbRt2kQsNLDSMeyCfiNmVX3/Z0YRJCeiC7jHeBK38dXAm97mOWY/NVdEXHnrpmtBaoC232bvnHOXedhpCP4jkifoPTKiOedcw94HKlcZnYG8DmQzC/r5rc75973LtXxmdlA4G/OuWFeZzkWM+tJ6YnoWGA9MNY5t9PbVEdnZvcCf6R0GWIZcLVzbp+3qX5hZv8FBlI62jgLuBuYC7wJtKJ03PsfnHNlTwAHXDlZJ+OH7oqI4hcRkV9ExFKPiIj8QsUvIhJhVPwiIhFGxS8iEmFU/CIiEUbFL3KCfNNJN5hZPd/jur7HCd4mE6kYFb/ICXLObQaeBqb6Nk0Fpjvn0jwLJXICdB2/yEnwjaxYCjwPXAP0dM4VeZtKpGLCZlaPSCA554rMbALwIXCBSl9CiZZ6RE7ehZSOI+7qdRCRE6HiFzkJvnk651P6DmS3+qY8ioQEFb/ICfJNonya0vch2ARMo/QNSURCgopf5MRdA2xyzi3wPX4K6GxmZ3uYSaTCdFWPiEiE0RG/iEiEUfGLiEQYFb+ISIRR8YuIRBgVv4hIhFHxi4hEGBW/iEiE+f8GUmrVt706/QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualizing X and y\n",
    "\n",
    "plt.plot(X,y) #This will plot a line\n",
    "plt.scatter(X,y) \n",
    "plt.annotate('y=2X+1', xy=(3,4))\n",
    "plt.xlabel('X')\n",
    "plt.ylabel('y')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "StLIYBqUu_pe",
   "metadata": {
    "id": "StLIYBqUu_pe"
   },
   "source": [
    "<a name='2-3'></a>\n",
    "### 2.3  Preparing data for the model\n",
    "\n",
    "Usually when working with real world datasets, we need to spend an enourmous amount of time preparing it. The type of things to be done depend on the kind of the dataset, but typically, it can be removing/filling missing values, scaling the features either with normalization or standardization, and so on. \n",
    "\n",
    "For now, we can leave our data as it is. In the next labs, we will see some real world datasets that needs extra work before feeding them to the machine learning model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "u8JV6HwXxcDM",
   "metadata": {
    "id": "u8JV6HwXxcDM"
   },
   "source": [
    "<a name='2-4'></a>\n",
    "### 2.4  Creating, Compiling and Training a Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "-SfBQQunx710",
   "metadata": {
    "id": "-SfBQQunx710"
   },
   "source": [
    "We are going to create the model having a one layer, one neuron(or unit) and as the input data is a single number, we will set the `input_shape` to `[1]`. \n",
    "\n",
    "Later, we will explain everything we did."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "sfadbvPlzFiO",
   "metadata": {
    "id": "sfadbvPlzFiO"
   },
   "outputs": [],
   "source": [
    "model = tf.keras.Sequential([\n",
    "                             \n",
    "                             keras.layers.Dense(units=1, input_shape=[1])\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Y-irJjCqzb0P",
   "metadata": {
    "id": "Y-irJjCqzb0P"
   },
   "source": [
    "And we can see the model summary. Model summary is essential for quick review of the model architecture. As you can see, we only have one dense layer, and 2 parameters(weight and bias)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "jqyr0pCwzeGy",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "jqyr0pCwzeGy",
    "outputId": "d5434f60-7266-431d-ef09-17245a633490"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense (Dense)                (None, 1)                 2         \n",
      "=================================================================\n",
      "Total params: 2\n",
      "Trainable params: 2\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "pt6VEiYxz-GI",
   "metadata": {
    "id": "pt6VEiYxz-GI"
   },
   "source": [
    "The model that we created is called a  Sequential model. We create it by adding one layer after another. If we had many layers, it would be like a sequence or series of layers, one after another, from the input to the ouput. You can learn more about Sequential API [here](https://keras.io/api/models/sequential/)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "EUdQUMUp1O47",
   "metadata": {
    "id": "EUdQUMUp1O47"
   },
   "source": [
    "After we have created the model, all we have is an empty graphs. We need to do two more things, compiling and training/fitting the model to the presented data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "-6HLWEe62CBC",
   "metadata": {
    "id": "-6HLWEe62CBC"
   },
   "outputs": [],
   "source": [
    "model.compile(optimizer='sgd', \n",
    "              loss='mean_squared_error',\n",
    "              metrics=['mse'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "GI77KEDn2Bac",
   "metadata": {
    "id": "GI77KEDn2Bac"
   },
   "source": [
    "The single most reason of why we compile the model is to specify the optimizer, loss function and the metrics that we want to track during training. \n",
    "\n",
    "During training, loss function will be used to measure the difference between the prediction and the actual output. Such difference is called `error`, but we want to measure the `the mean of squared error`, hence the name.\n",
    "\n",
    "`Error = Actual value – Predicted value`. \n",
    "\n",
    "On the otherhand, optimizer is used to reduce the error between the actual output and predicted value. The optimizer will make continous guesses until the minimum error is reached. There are many optimizers, but for now let's use SGD (Stockastic Gradient Descent). In later labs, we will explore other optimizers. \n",
    "\n",
    "By fitting the model to the data, here are what happen:\n",
    "\n",
    "* The model iterate through each input data point and estimate prediction \n",
    "* The difference between the actual and predicted value or error is calculated by loss function\n",
    "* The error is minimized by optimizer as we go through the data\n",
    "* The above iteration continue until the number of epochs are reached. \n",
    "\n",
    "\n",
    "Let's see that in action, fitting the model to the data, calculating and reducing the error until we make 500 turns. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "pJ8gCQVK4CbR",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "pJ8gCQVK4CbR",
    "outputId": "63df70a9-94e5-49d6-b655-c20aee929dca"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/500\n",
      "1/1 [==============================] - 1s 583ms/step - loss: 240.2002 - mse: 240.2002\n",
      "Epoch 2/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 20.7756 - mse: 20.7756\n",
      "Epoch 3/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 1.9780 - mse: 1.9780\n",
      "Epoch 4/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 0.3642 - mse: 0.3642\n",
      "Epoch 5/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 0.2222 - mse: 0.2222\n",
      "Epoch 6/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.2064 - mse: 0.2064\n",
      "Epoch 7/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.2015 - mse: 0.2015\n",
      "Epoch 8/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.1976 - mse: 0.1976\n",
      "Epoch 9/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 0.1938 - mse: 0.1938\n",
      "Epoch 10/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.1901 - mse: 0.1901\n",
      "Epoch 11/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.1865 - mse: 0.1865\n",
      "Epoch 12/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.1829 - mse: 0.1829\n",
      "Epoch 13/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 0.1794 - mse: 0.1794\n",
      "Epoch 14/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 0.1760 - mse: 0.1760\n",
      "Epoch 15/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 0.1726 - mse: 0.1726\n",
      "Epoch 16/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.1693 - mse: 0.1693\n",
      "Epoch 17/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.1661 - mse: 0.1661\n",
      "Epoch 18/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.1630 - mse: 0.1630\n",
      "Epoch 19/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.1598 - mse: 0.1598\n",
      "Epoch 20/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.1568 - mse: 0.1568\n",
      "Epoch 21/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.1538 - mse: 0.1538\n",
      "Epoch 22/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.1509 - mse: 0.1509\n",
      "Epoch 23/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.1480 - mse: 0.1480\n",
      "Epoch 24/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.1452 - mse: 0.1452\n",
      "Epoch 25/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.1424 - mse: 0.1424\n",
      "Epoch 26/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 0.1397 - mse: 0.1397\n",
      "Epoch 27/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.1370 - mse: 0.1370\n",
      "Epoch 28/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 0.1344 - mse: 0.1344\n",
      "Epoch 29/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.1318 - mse: 0.1318\n",
      "Epoch 30/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 0.1293 - mse: 0.1293\n",
      "Epoch 31/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.1269 - mse: 0.1269\n",
      "Epoch 32/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.1244 - mse: 0.1244\n",
      "Epoch 33/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.1221 - mse: 0.1221\n",
      "Epoch 34/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.1197 - mse: 0.1197\n",
      "Epoch 35/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.1175 - mse: 0.1175\n",
      "Epoch 36/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.1152 - mse: 0.1152\n",
      "Epoch 37/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.1130 - mse: 0.1130\n",
      "Epoch 38/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 0.1109 - mse: 0.1109\n",
      "Epoch 39/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.1087 - mse: 0.1087\n",
      "Epoch 40/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.1067 - mse: 0.1067\n",
      "Epoch 41/500\n",
      "1/1 [==============================] - 0s 20ms/step - loss: 0.1046 - mse: 0.1046\n",
      "Epoch 42/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.1026 - mse: 0.1026\n",
      "Epoch 43/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.1007 - mse: 0.1007\n",
      "Epoch 44/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0988 - mse: 0.0988\n",
      "Epoch 45/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0969 - mse: 0.0969\n",
      "Epoch 46/500\n",
      "1/1 [==============================] - 0s 19ms/step - loss: 0.0950 - mse: 0.0950\n",
      "Epoch 47/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0932 - mse: 0.0932\n",
      "Epoch 48/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0914 - mse: 0.0914\n",
      "Epoch 49/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0897 - mse: 0.0897\n",
      "Epoch 50/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0880 - mse: 0.0880\n",
      "Epoch 51/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0863 - mse: 0.0863\n",
      "Epoch 52/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 0.0847 - mse: 0.0847\n",
      "Epoch 53/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0830 - mse: 0.0830\n",
      "Epoch 54/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0815 - mse: 0.0815\n",
      "Epoch 55/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0799 - mse: 0.0799\n",
      "Epoch 56/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0784 - mse: 0.0784\n",
      "Epoch 57/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0769 - mse: 0.0769\n",
      "Epoch 58/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 0.0754 - mse: 0.0754\n",
      "Epoch 59/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0740 - mse: 0.0740\n",
      "Epoch 60/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0726 - mse: 0.0726\n",
      "Epoch 61/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0712 - mse: 0.0712\n",
      "Epoch 62/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0698 - mse: 0.0698\n",
      "Epoch 63/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0685 - mse: 0.0685\n",
      "Epoch 64/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0672 - mse: 0.0672\n",
      "Epoch 65/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0659 - mse: 0.0659\n",
      "Epoch 66/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0647 - mse: 0.0647\n",
      "Epoch 67/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0634 - mse: 0.0634\n",
      "Epoch 68/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0622 - mse: 0.0622\n",
      "Epoch 69/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0610 - mse: 0.0610\n",
      "Epoch 70/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0599 - mse: 0.0599\n",
      "Epoch 71/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0587 - mse: 0.0587\n",
      "Epoch 72/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0576 - mse: 0.0576\n",
      "Epoch 73/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0565 - mse: 0.0565\n",
      "Epoch 74/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0554 - mse: 0.0554\n",
      "Epoch 75/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0544 - mse: 0.0544\n",
      "Epoch 76/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0533 - mse: 0.0533\n",
      "Epoch 77/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0523 - mse: 0.0523\n",
      "Epoch 78/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0513 - mse: 0.0513\n",
      "Epoch 79/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0503 - mse: 0.0503\n",
      "Epoch 80/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0494 - mse: 0.0494\n",
      "Epoch 81/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0484 - mse: 0.0484\n",
      "Epoch 82/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0475 - mse: 0.0475\n",
      "Epoch 83/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0466 - mse: 0.0466\n",
      "Epoch 84/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0457 - mse: 0.0457\n",
      "Epoch 85/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0448 - mse: 0.0448\n",
      "Epoch 86/500\n",
      "1/1 [==============================] - 0s 13ms/step - loss: 0.0440 - mse: 0.0440\n",
      "Epoch 87/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 0.0431 - mse: 0.0431\n",
      "Epoch 88/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0423 - mse: 0.0423\n",
      "Epoch 89/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0415 - mse: 0.0415\n",
      "Epoch 90/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 0.0407 - mse: 0.0407\n",
      "Epoch 91/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0399 - mse: 0.0399\n",
      "Epoch 92/500\n",
      "1/1 [==============================] - 0s 14ms/step - loss: 0.0392 - mse: 0.0392\n",
      "Epoch 93/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0384 - mse: 0.0384\n",
      "Epoch 94/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0377 - mse: 0.0377\n",
      "Epoch 95/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0370 - mse: 0.0370\n",
      "Epoch 96/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0363 - mse: 0.0363\n",
      "Epoch 97/500\n",
      "1/1 [==============================] - 0s 16ms/step - loss: 0.0356 - mse: 0.0356\n",
      "Epoch 98/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0349 - mse: 0.0349\n",
      "Epoch 99/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0342 - mse: 0.0342\n",
      "Epoch 100/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 0.0336 - mse: 0.0336\n",
      "Epoch 101/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0329 - mse: 0.0329\n",
      "Epoch 102/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0323 - mse: 0.0323\n",
      "Epoch 103/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 0.0317 - mse: 0.0317\n",
      "Epoch 104/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0311 - mse: 0.0311\n",
      "Epoch 105/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0305 - mse: 0.0305\n",
      "Epoch 106/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 0.0299 - mse: 0.0299\n",
      "Epoch 107/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0294 - mse: 0.0294\n",
      "Epoch 108/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0288 - mse: 0.0288\n",
      "Epoch 109/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0282 - mse: 0.0282\n",
      "Epoch 110/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0277 - mse: 0.0277\n",
      "Epoch 111/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 0.0272 - mse: 0.0272\n",
      "Epoch 112/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 0.0267 - mse: 0.0267\n",
      "Epoch 113/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0261 - mse: 0.0261\n",
      "Epoch 114/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0257 - mse: 0.0257\n",
      "Epoch 115/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0252 - mse: 0.0252\n",
      "Epoch 116/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 0.0247 - mse: 0.0247\n",
      "Epoch 117/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0242 - mse: 0.0242\n",
      "Epoch 118/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0237 - mse: 0.0237\n",
      "Epoch 119/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0233 - mse: 0.0233\n",
      "Epoch 120/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0229 - mse: 0.0229\n",
      "Epoch 121/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0224 - mse: 0.0224\n",
      "Epoch 122/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0220 - mse: 0.0220\n",
      "Epoch 123/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0216 - mse: 0.0216\n",
      "Epoch 124/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0212 - mse: 0.0212\n",
      "Epoch 125/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0208 - mse: 0.0208\n",
      "Epoch 126/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0204 - mse: 0.0204\n",
      "Epoch 127/500\n",
      "1/1 [==============================] - 0s 15ms/step - loss: 0.0200 - mse: 0.0200\n",
      "Epoch 128/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0196 - mse: 0.0196\n",
      "Epoch 129/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 0.0192 - mse: 0.0192\n",
      "Epoch 130/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0188 - mse: 0.0188\n",
      "Epoch 131/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0185 - mse: 0.0185\n",
      "Epoch 132/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 0.0181 - mse: 0.0181\n",
      "Epoch 133/500\n",
      "1/1 [==============================] - 0s 15ms/step - loss: 0.0178 - mse: 0.0178\n",
      "Epoch 134/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0175 - mse: 0.0175\n",
      "Epoch 135/500\n",
      "1/1 [==============================] - 0s 13ms/step - loss: 0.0171 - mse: 0.0171\n",
      "Epoch 136/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 0.0168 - mse: 0.0168\n",
      "Epoch 137/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0165 - mse: 0.0165\n",
      "Epoch 138/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0162 - mse: 0.0162\n",
      "Epoch 139/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0158 - mse: 0.0158\n",
      "Epoch 140/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0155 - mse: 0.0155\n",
      "Epoch 141/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0152 - mse: 0.0152\n",
      "Epoch 142/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0150 - mse: 0.0150\n",
      "Epoch 143/500\n",
      "1/1 [==============================] - 0s 15ms/step - loss: 0.0147 - mse: 0.0147\n",
      "Epoch 144/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 0.0144 - mse: 0.0144\n",
      "Epoch 145/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0141 - mse: 0.0141\n",
      "Epoch 146/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0138 - mse: 0.0138\n",
      "Epoch 147/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0136 - mse: 0.0136\n",
      "Epoch 148/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 0.0133 - mse: 0.0133\n",
      "Epoch 149/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0131 - mse: 0.0131\n",
      "Epoch 150/500\n",
      "1/1 [==============================] - 0s 13ms/step - loss: 0.0128 - mse: 0.0128\n",
      "Epoch 151/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0126 - mse: 0.0126\n",
      "Epoch 152/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0123 - mse: 0.0123\n",
      "Epoch 153/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0121 - mse: 0.0121\n",
      "Epoch 154/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 0.0119 - mse: 0.0119\n",
      "Epoch 155/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0116 - mse: 0.0116\n",
      "Epoch 156/500\n",
      "1/1 [==============================] - 0s 14ms/step - loss: 0.0114 - mse: 0.0114\n",
      "Epoch 157/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 0.0112 - mse: 0.0112\n",
      "Epoch 158/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0110 - mse: 0.0110\n",
      "Epoch 159/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0108 - mse: 0.0108\n",
      "Epoch 160/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0106 - mse: 0.0106\n",
      "Epoch 161/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0104 - mse: 0.0104\n",
      "Epoch 162/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0102 - mse: 0.0102\n",
      "Epoch 163/500\n",
      "1/1 [==============================] - 0s 14ms/step - loss: 0.0100 - mse: 0.0100\n",
      "Epoch 164/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0098 - mse: 0.0098\n",
      "Epoch 165/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0096 - mse: 0.0096\n",
      "Epoch 166/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0094 - mse: 0.0094\n",
      "Epoch 167/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0092 - mse: 0.0092\n",
      "Epoch 168/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0091 - mse: 0.0091\n",
      "Epoch 169/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0089 - mse: 0.0089\n",
      "Epoch 170/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0087 - mse: 0.0087\n",
      "Epoch 171/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0086 - mse: 0.0086\n",
      "Epoch 172/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 0.0084 - mse: 0.0084\n",
      "Epoch 173/500\n",
      "1/1 [==============================] - 0s 17ms/step - loss: 0.0082 - mse: 0.0082\n",
      "Epoch 174/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0081 - mse: 0.0081\n",
      "Epoch 175/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0079 - mse: 0.0079\n",
      "Epoch 176/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 0.0078 - mse: 0.0078\n",
      "Epoch 177/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0076 - mse: 0.0076\n",
      "Epoch 178/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0075 - mse: 0.0075\n",
      "Epoch 179/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0073 - mse: 0.0073\n",
      "Epoch 180/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0072 - mse: 0.0072\n",
      "Epoch 181/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0071 - mse: 0.0071\n",
      "Epoch 182/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0069 - mse: 0.0069\n",
      "Epoch 183/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0068 - mse: 0.0068\n",
      "Epoch 184/500\n",
      "1/1 [==============================] - 0s 14ms/step - loss: 0.0067 - mse: 0.0067\n",
      "Epoch 185/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0065 - mse: 0.0065\n",
      "Epoch 186/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0064 - mse: 0.0064\n",
      "Epoch 187/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 0.0063 - mse: 0.0063\n",
      "Epoch 188/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0062 - mse: 0.0062\n",
      "Epoch 189/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0061 - mse: 0.0061\n",
      "Epoch 190/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0059 - mse: 0.0059\n",
      "Epoch 191/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0058 - mse: 0.0058\n",
      "Epoch 192/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0057 - mse: 0.0057\n",
      "Epoch 193/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0056 - mse: 0.0056\n",
      "Epoch 194/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0055 - mse: 0.0055\n",
      "Epoch 195/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 0.0054 - mse: 0.0054\n",
      "Epoch 196/500\n",
      "1/1 [==============================] - 0s 23ms/step - loss: 0.0053 - mse: 0.0053\n",
      "Epoch 197/500\n",
      "1/1 [==============================] - 0s 16ms/step - loss: 0.0052 - mse: 0.0052\n",
      "Epoch 198/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 0.0051 - mse: 0.0051\n",
      "Epoch 199/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 0.0050 - mse: 0.0050\n",
      "Epoch 200/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0049 - mse: 0.0049\n",
      "Epoch 201/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 0.0048 - mse: 0.0048\n",
      "Epoch 202/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 0.0047 - mse: 0.0047\n",
      "Epoch 203/500\n",
      "1/1 [==============================] - 0s 16ms/step - loss: 0.0046 - mse: 0.0046\n",
      "Epoch 204/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0045 - mse: 0.0045\n",
      "Epoch 205/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0044 - mse: 0.0044\n",
      "Epoch 206/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0044 - mse: 0.0044\n",
      "Epoch 207/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0043 - mse: 0.0043\n",
      "Epoch 208/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0042 - mse: 0.0042\n",
      "Epoch 209/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0041 - mse: 0.0041\n",
      "Epoch 210/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0040 - mse: 0.0040\n",
      "Epoch 211/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0040 - mse: 0.0040\n",
      "Epoch 212/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0039 - mse: 0.0039\n",
      "Epoch 213/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0038 - mse: 0.0038\n",
      "Epoch 214/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0037 - mse: 0.0037\n",
      "Epoch 215/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0037 - mse: 0.0037\n",
      "Epoch 216/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0036 - mse: 0.0036\n",
      "Epoch 217/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0035 - mse: 0.0035\n",
      "Epoch 218/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 0.0035 - mse: 0.0035\n",
      "Epoch 219/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0034 - mse: 0.0034\n",
      "Epoch 220/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0033 - mse: 0.0033\n",
      "Epoch 221/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0033 - mse: 0.0033\n",
      "Epoch 222/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0032 - mse: 0.0032\n",
      "Epoch 223/500\n",
      "1/1 [==============================] - 0s 13ms/step - loss: 0.0031 - mse: 0.0031\n",
      "Epoch 224/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0031 - mse: 0.0031\n",
      "Epoch 225/500\n",
      "1/1 [==============================] - 0s 18ms/step - loss: 0.0030 - mse: 0.0030\n",
      "Epoch 226/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0030 - mse: 0.0030\n",
      "Epoch 227/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0029 - mse: 0.0029\n",
      "Epoch 228/500\n",
      "1/1 [==============================] - 0s 23ms/step - loss: 0.0029 - mse: 0.0029\n",
      "Epoch 229/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0028 - mse: 0.0028\n",
      "Epoch 230/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0027 - mse: 0.0027\n",
      "Epoch 231/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0027 - mse: 0.0027\n",
      "Epoch 232/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0026 - mse: 0.0026\n",
      "Epoch 233/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0026 - mse: 0.0026\n",
      "Epoch 234/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0025 - mse: 0.0025\n",
      "Epoch 235/500\n",
      "1/1 [==============================] - 0s 18ms/step - loss: 0.0025 - mse: 0.0025\n",
      "Epoch 236/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 0.0024 - mse: 0.0024\n",
      "Epoch 237/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0024 - mse: 0.0024\n",
      "Epoch 238/500\n",
      "1/1 [==============================] - 0s 24ms/step - loss: 0.0024 - mse: 0.0024\n",
      "Epoch 239/500\n",
      "1/1 [==============================] - 0s 29ms/step - loss: 0.0023 - mse: 0.0023\n",
      "Epoch 240/500\n",
      "1/1 [==============================] - 0s 24ms/step - loss: 0.0023 - mse: 0.0023\n",
      "Epoch 241/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0022 - mse: 0.0022\n",
      "Epoch 242/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0022 - mse: 0.0022\n",
      "Epoch 243/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0021 - mse: 0.0021\n",
      "Epoch 244/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0021 - mse: 0.0021\n",
      "Epoch 245/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0021 - mse: 0.0021\n",
      "Epoch 246/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0020 - mse: 0.0020\n",
      "Epoch 247/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 0.0020 - mse: 0.0020\n",
      "Epoch 248/500\n",
      "1/1 [==============================] - 0s 13ms/step - loss: 0.0019 - mse: 0.0019\n",
      "Epoch 249/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0019 - mse: 0.0019\n",
      "Epoch 250/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0019 - mse: 0.0019\n",
      "Epoch 251/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0018 - mse: 0.0018\n",
      "Epoch 252/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0018 - mse: 0.0018\n",
      "Epoch 253/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0018 - mse: 0.0018\n",
      "Epoch 254/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 0.0017 - mse: 0.0017\n",
      "Epoch 255/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0017 - mse: 0.0017\n",
      "Epoch 256/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0017 - mse: 0.0017\n",
      "Epoch 257/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0016 - mse: 0.0016\n",
      "Epoch 258/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0016 - mse: 0.0016\n",
      "Epoch 259/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0016 - mse: 0.0016\n",
      "Epoch 260/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0015 - mse: 0.0015\n",
      "Epoch 261/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0015 - mse: 0.0015\n",
      "Epoch 262/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0015 - mse: 0.0015\n",
      "Epoch 263/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 0.0015 - mse: 0.0015\n",
      "Epoch 264/500\n",
      "1/1 [==============================] - 0s 14ms/step - loss: 0.0014 - mse: 0.0014\n",
      "Epoch 265/500\n",
      "1/1 [==============================] - 0s 21ms/step - loss: 0.0014 - mse: 0.0014\n",
      "Epoch 266/500\n",
      "1/1 [==============================] - 0s 30ms/step - loss: 0.0014 - mse: 0.0014\n",
      "Epoch 267/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0013 - mse: 0.0013\n",
      "Epoch 268/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0013 - mse: 0.0013\n",
      "Epoch 269/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 0.0013 - mse: 0.0013\n",
      "Epoch 270/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0013 - mse: 0.0013\n",
      "Epoch 271/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0012 - mse: 0.0012\n",
      "Epoch 272/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0012 - mse: 0.0012\n",
      "Epoch 273/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 0.0012 - mse: 0.0012\n",
      "Epoch 274/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0012 - mse: 0.0012\n",
      "Epoch 275/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0012 - mse: 0.0012\n",
      "Epoch 276/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0011 - mse: 0.0011\n",
      "Epoch 277/500\n",
      "1/1 [==============================] - 0s 17ms/step - loss: 0.0011 - mse: 0.0011\n",
      "Epoch 278/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 0.0011 - mse: 0.0011\n",
      "Epoch 279/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0011 - mse: 0.0011\n",
      "Epoch 280/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 0.0010 - mse: 0.0010\n",
      "Epoch 281/500\n",
      "1/1 [==============================] - 0s 13ms/step - loss: 0.0010 - mse: 0.0010\n",
      "Epoch 282/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 0.0010 - mse: 0.0010\n",
      "Epoch 283/500\n",
      "1/1 [==============================] - 0s 14ms/step - loss: 9.8983e-04 - mse: 9.8983e-04\n",
      "Epoch 284/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 9.7094e-04 - mse: 9.7094e-04\n",
      "Epoch 285/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 9.5243e-04 - mse: 9.5243e-04\n",
      "Epoch 286/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 9.3426e-04 - mse: 9.3426e-04\n",
      "Epoch 287/500\n",
      "1/1 [==============================] - 0s 17ms/step - loss: 9.1644e-04 - mse: 9.1644e-04\n",
      "Epoch 288/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 8.9895e-04 - mse: 8.9895e-04\n",
      "Epoch 289/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 8.8180e-04 - mse: 8.8180e-04\n",
      "Epoch 290/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 8.6499e-04 - mse: 8.6499e-04\n",
      "Epoch 291/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 8.4849e-04 - mse: 8.4849e-04\n",
      "Epoch 292/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 8.3230e-04 - mse: 8.3230e-04\n",
      "Epoch 293/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 8.1643e-04 - mse: 8.1643e-04\n",
      "Epoch 294/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 8.0086e-04 - mse: 8.0086e-04\n",
      "Epoch 295/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 7.8558e-04 - mse: 7.8558e-04\n",
      "Epoch 296/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 7.7058e-04 - mse: 7.7058e-04\n",
      "Epoch 297/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 7.5588e-04 - mse: 7.5588e-04\n",
      "Epoch 298/500\n",
      "1/1 [==============================] - 0s 15ms/step - loss: 7.4147e-04 - mse: 7.4147e-04\n",
      "Epoch 299/500\n",
      "1/1 [==============================] - 0s 22ms/step - loss: 7.2734e-04 - mse: 7.2734e-04\n",
      "Epoch 300/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 7.1345e-04 - mse: 7.1345e-04\n",
      "Epoch 301/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 6.9984e-04 - mse: 6.9984e-04\n",
      "Epoch 302/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 6.8650e-04 - mse: 6.8650e-04\n",
      "Epoch 303/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 6.7341e-04 - mse: 6.7341e-04\n",
      "Epoch 304/500\n",
      "1/1 [==============================] - 0s 68ms/step - loss: 6.6056e-04 - mse: 6.6056e-04\n",
      "Epoch 305/500\n",
      "1/1 [==============================] - 0s 17ms/step - loss: 6.4796e-04 - mse: 6.4796e-04\n",
      "Epoch 306/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 6.3560e-04 - mse: 6.3560e-04\n",
      "Epoch 307/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 6.2348e-04 - mse: 6.2348e-04\n",
      "Epoch 308/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 6.1158e-04 - mse: 6.1158e-04\n",
      "Epoch 309/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 5.9992e-04 - mse: 5.9992e-04\n",
      "Epoch 310/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 5.8847e-04 - mse: 5.8847e-04\n",
      "Epoch 311/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 5.7725e-04 - mse: 5.7725e-04\n",
      "Epoch 312/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 5.6624e-04 - mse: 5.6624e-04\n",
      "Epoch 313/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 5.5544e-04 - mse: 5.5544e-04\n",
      "Epoch 314/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 5.4484e-04 - mse: 5.4484e-04\n",
      "Epoch 315/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 5.3445e-04 - mse: 5.3445e-04\n",
      "Epoch 316/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 5.2425e-04 - mse: 5.2425e-04\n",
      "Epoch 317/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 5.1425e-04 - mse: 5.1425e-04\n",
      "Epoch 318/500\n",
      "1/1 [==============================] - 0s 18ms/step - loss: 5.0444e-04 - mse: 5.0444e-04\n",
      "Epoch 319/500\n",
      "1/1 [==============================] - 0s 18ms/step - loss: 4.9482e-04 - mse: 4.9482e-04\n",
      "Epoch 320/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 4.8538e-04 - mse: 4.8538e-04\n",
      "Epoch 321/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 4.7613e-04 - mse: 4.7613e-04\n",
      "Epoch 322/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 4.6704e-04 - mse: 4.6704e-04\n",
      "Epoch 323/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 4.5813e-04 - mse: 4.5813e-04\n",
      "Epoch 324/500\n",
      "1/1 [==============================] - 0s 18ms/step - loss: 4.4940e-04 - mse: 4.4940e-04\n",
      "Epoch 325/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 4.4082e-04 - mse: 4.4082e-04\n",
      "Epoch 326/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 4.3242e-04 - mse: 4.3242e-04\n",
      "Epoch 327/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 4.2417e-04 - mse: 4.2417e-04\n",
      "Epoch 328/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 4.1607e-04 - mse: 4.1607e-04\n",
      "Epoch 329/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 4.0814e-04 - mse: 4.0814e-04\n",
      "Epoch 330/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 4.0036e-04 - mse: 4.0036e-04\n",
      "Epoch 331/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 3.9272e-04 - mse: 3.9272e-04\n",
      "Epoch 332/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 3.8523e-04 - mse: 3.8523e-04\n",
      "Epoch 333/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 3.7788e-04 - mse: 3.7788e-04\n",
      "Epoch 334/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 3.7067e-04 - mse: 3.7067e-04\n",
      "Epoch 335/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 3.6360e-04 - mse: 3.6360e-04\n",
      "Epoch 336/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 3.5666e-04 - mse: 3.5666e-04\n",
      "Epoch 337/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 3.4985e-04 - mse: 3.4985e-04\n",
      "Epoch 338/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 3.4318e-04 - mse: 3.4318e-04\n",
      "Epoch 339/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 3.3664e-04 - mse: 3.3664e-04\n",
      "Epoch 340/500\n",
      "1/1 [==============================] - 0s 14ms/step - loss: 3.3022e-04 - mse: 3.3022e-04\n",
      "Epoch 341/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 3.2392e-04 - mse: 3.2392e-04\n",
      "Epoch 342/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 3.1774e-04 - mse: 3.1774e-04\n",
      "Epoch 343/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 3.1168e-04 - mse: 3.1168e-04\n",
      "Epoch 344/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 3.0573e-04 - mse: 3.0573e-04\n",
      "Epoch 345/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 2.9990e-04 - mse: 2.9990e-04\n",
      "Epoch 346/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 2.9418e-04 - mse: 2.9418e-04\n",
      "Epoch 347/500\n",
      "1/1 [==============================] - 0s 16ms/step - loss: 2.8857e-04 - mse: 2.8857e-04\n",
      "Epoch 348/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 2.8307e-04 - mse: 2.8307e-04\n",
      "Epoch 349/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 2.7767e-04 - mse: 2.7767e-04\n",
      "Epoch 350/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 2.7237e-04 - mse: 2.7237e-04\n",
      "Epoch 351/500\n",
      "1/1 [==============================] - 0s 51ms/step - loss: 2.6718e-04 - mse: 2.6718e-04\n",
      "Epoch 352/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 2.6208e-04 - mse: 2.6208e-04\n",
      "Epoch 353/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 2.5708e-04 - mse: 2.5708e-04\n",
      "Epoch 354/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 2.5218e-04 - mse: 2.5218e-04\n",
      "Epoch 355/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 2.4736e-04 - mse: 2.4736e-04\n",
      "Epoch 356/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 2.4264e-04 - mse: 2.4264e-04\n",
      "Epoch 357/500\n",
      "1/1 [==============================] - 0s 14ms/step - loss: 2.3802e-04 - mse: 2.3802e-04\n",
      "Epoch 358/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 2.3348e-04 - mse: 2.3348e-04\n",
      "Epoch 359/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 2.2903e-04 - mse: 2.2903e-04\n",
      "Epoch 360/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 2.2465e-04 - mse: 2.2465e-04\n",
      "Epoch 361/500\n",
      "1/1 [==============================] - 0s 15ms/step - loss: 2.2037e-04 - mse: 2.2037e-04\n",
      "Epoch 362/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 2.1617e-04 - mse: 2.1617e-04\n",
      "Epoch 363/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 2.1205e-04 - mse: 2.1205e-04\n",
      "Epoch 364/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 2.0800e-04 - mse: 2.0800e-04\n",
      "Epoch 365/500\n",
      "1/1 [==============================] - 0s 13ms/step - loss: 2.0403e-04 - mse: 2.0403e-04\n",
      "Epoch 366/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 2.0014e-04 - mse: 2.0014e-04\n",
      "Epoch 367/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 1.9632e-04 - mse: 1.9632e-04\n",
      "Epoch 368/500\n",
      "1/1 [==============================] - 0s 19ms/step - loss: 1.9258e-04 - mse: 1.9258e-04\n",
      "Epoch 369/500\n",
      "1/1 [==============================] - 0s 13ms/step - loss: 1.8891e-04 - mse: 1.8891e-04\n",
      "Epoch 370/500\n",
      "1/1 [==============================] - 0s 15ms/step - loss: 1.8530e-04 - mse: 1.8530e-04\n",
      "Epoch 371/500\n",
      "1/1 [==============================] - 0s 13ms/step - loss: 1.8177e-04 - mse: 1.8177e-04\n",
      "Epoch 372/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 1.7830e-04 - mse: 1.7830e-04\n",
      "Epoch 373/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 1.7490e-04 - mse: 1.7490e-04\n",
      "Epoch 374/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 1.7156e-04 - mse: 1.7156e-04\n",
      "Epoch 375/500\n",
      "1/1 [==============================] - 0s 13ms/step - loss: 1.6829e-04 - mse: 1.6829e-04\n",
      "Epoch 376/500\n",
      "1/1 [==============================] - 0s 22ms/step - loss: 1.6508e-04 - mse: 1.6508e-04\n",
      "Epoch 377/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 1.6193e-04 - mse: 1.6193e-04\n",
      "Epoch 378/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 1.5884e-04 - mse: 1.5884e-04\n",
      "Epoch 379/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 1.5581e-04 - mse: 1.5581e-04\n",
      "Epoch 380/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 1.5284e-04 - mse: 1.5284e-04\n",
      "Epoch 381/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 1.4993e-04 - mse: 1.4993e-04\n",
      "Epoch 382/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 1.4706e-04 - mse: 1.4706e-04\n",
      "Epoch 383/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 1.4426e-04 - mse: 1.4426e-04\n",
      "Epoch 384/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 1.4151e-04 - mse: 1.4151e-04\n",
      "Epoch 385/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 1.3881e-04 - mse: 1.3881e-04\n",
      "Epoch 386/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 1.3616e-04 - mse: 1.3616e-04\n",
      "Epoch 387/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 1.3356e-04 - mse: 1.3356e-04\n",
      "Epoch 388/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 1.3102e-04 - mse: 1.3102e-04\n",
      "Epoch 389/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 1.2852e-04 - mse: 1.2852e-04\n",
      "Epoch 390/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 1.2607e-04 - mse: 1.2607e-04\n",
      "Epoch 391/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 1.2366e-04 - mse: 1.2366e-04\n",
      "Epoch 392/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 1.2130e-04 - mse: 1.2130e-04\n",
      "Epoch 393/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 1.1899e-04 - mse: 1.1899e-04\n",
      "Epoch 394/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 1.1672e-04 - mse: 1.1672e-04\n",
      "Epoch 395/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 1.1449e-04 - mse: 1.1449e-04\n",
      "Epoch 396/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 1.1231e-04 - mse: 1.1231e-04\n",
      "Epoch 397/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 1.1016e-04 - mse: 1.1016e-04\n",
      "Epoch 398/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 1.0807e-04 - mse: 1.0807e-04\n",
      "Epoch 399/500\n",
      "1/1 [==============================] - 0s 14ms/step - loss: 1.0600e-04 - mse: 1.0600e-04\n",
      "Epoch 400/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 1.0398e-04 - mse: 1.0398e-04\n",
      "Epoch 401/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 1.0200e-04 - mse: 1.0200e-04\n",
      "Epoch 402/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 1.0005e-04 - mse: 1.0005e-04\n",
      "Epoch 403/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 9.8142e-05 - mse: 9.8142e-05\n",
      "Epoch 404/500\n",
      "1/1 [==============================] - 0s 13ms/step - loss: 9.6271e-05 - mse: 9.6271e-05\n",
      "Epoch 405/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 9.4433e-05 - mse: 9.4433e-05\n",
      "Epoch 406/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 9.2632e-05 - mse: 9.2632e-05\n",
      "Epoch 407/500\n",
      "1/1 [==============================] - 0s 33ms/step - loss: 9.0865e-05 - mse: 9.0865e-05\n",
      "Epoch 408/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 8.9132e-05 - mse: 8.9132e-05\n",
      "Epoch 409/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 8.7431e-05 - mse: 8.7431e-05\n",
      "Epoch 410/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 8.5761e-05 - mse: 8.5761e-05\n",
      "Epoch 411/500\n",
      "1/1 [==============================] - 0s 15ms/step - loss: 8.4128e-05 - mse: 8.4128e-05\n",
      "Epoch 412/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 8.2522e-05 - mse: 8.2522e-05\n",
      "Epoch 413/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 8.0950e-05 - mse: 8.0950e-05\n",
      "Epoch 414/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 7.9403e-05 - mse: 7.9403e-05\n",
      "Epoch 415/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 7.7892e-05 - mse: 7.7892e-05\n",
      "Epoch 416/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 7.6404e-05 - mse: 7.6404e-05\n",
      "Epoch 417/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 7.4948e-05 - mse: 7.4948e-05\n",
      "Epoch 418/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 7.3519e-05 - mse: 7.3519e-05\n",
      "Epoch 419/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 7.2114e-05 - mse: 7.2114e-05\n",
      "Epoch 420/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 7.0738e-05 - mse: 7.0738e-05\n",
      "Epoch 421/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 6.9390e-05 - mse: 6.9390e-05\n",
      "Epoch 422/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 6.8068e-05 - mse: 6.8068e-05\n",
      "Epoch 423/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 6.6769e-05 - mse: 6.6769e-05\n",
      "Epoch 424/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 6.5493e-05 - mse: 6.5493e-05\n",
      "Epoch 425/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 6.4246e-05 - mse: 6.4246e-05\n",
      "Epoch 426/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 6.3018e-05 - mse: 6.3018e-05\n",
      "Epoch 427/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 6.1819e-05 - mse: 6.1819e-05\n",
      "Epoch 428/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 6.0638e-05 - mse: 6.0638e-05\n",
      "Epoch 429/500\n",
      "1/1 [==============================] - 0s 19ms/step - loss: 5.9481e-05 - mse: 5.9481e-05\n",
      "Epoch 430/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 5.8347e-05 - mse: 5.8347e-05\n",
      "Epoch 431/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 5.7234e-05 - mse: 5.7234e-05\n",
      "Epoch 432/500\n",
      "1/1 [==============================] - 0s 15ms/step - loss: 5.6141e-05 - mse: 5.6141e-05\n",
      "Epoch 433/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 5.5071e-05 - mse: 5.5071e-05\n",
      "Epoch 434/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 5.4021e-05 - mse: 5.4021e-05\n",
      "Epoch 435/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 5.2991e-05 - mse: 5.2991e-05\n",
      "Epoch 436/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 5.1979e-05 - mse: 5.1979e-05\n",
      "Epoch 437/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 5.0989e-05 - mse: 5.0989e-05\n",
      "Epoch 438/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 5.0018e-05 - mse: 5.0018e-05\n",
      "Epoch 439/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 4.9064e-05 - mse: 4.9064e-05\n",
      "Epoch 440/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 4.8127e-05 - mse: 4.8127e-05\n",
      "Epoch 441/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 4.7209e-05 - mse: 4.7209e-05\n",
      "Epoch 442/500\n",
      "1/1 [==============================] - 0s 24ms/step - loss: 4.6309e-05 - mse: 4.6309e-05\n",
      "Epoch 443/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 4.5426e-05 - mse: 4.5426e-05\n",
      "Epoch 444/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 4.4558e-05 - mse: 4.4558e-05\n",
      "Epoch 445/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 4.3708e-05 - mse: 4.3708e-05\n",
      "Epoch 446/500\n",
      "1/1 [==============================] - 0s 10ms/step - loss: 4.2875e-05 - mse: 4.2875e-05\n",
      "Epoch 447/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 4.2056e-05 - mse: 4.2056e-05\n",
      "Epoch 448/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 4.1255e-05 - mse: 4.1255e-05\n",
      "Epoch 449/500\n",
      "1/1 [==============================] - 0s 16ms/step - loss: 4.0467e-05 - mse: 4.0467e-05\n",
      "Epoch 450/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 3.9696e-05 - mse: 3.9696e-05\n",
      "Epoch 451/500\n",
      "1/1 [==============================] - 0s 20ms/step - loss: 3.8938e-05 - mse: 3.8938e-05\n",
      "Epoch 452/500\n",
      "1/1 [==============================] - 0s 17ms/step - loss: 3.8195e-05 - mse: 3.8195e-05\n",
      "Epoch 453/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 3.7467e-05 - mse: 3.7467e-05\n",
      "Epoch 454/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 3.6752e-05 - mse: 3.6752e-05\n",
      "Epoch 455/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 3.6050e-05 - mse: 3.6050e-05\n",
      "Epoch 456/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 3.5364e-05 - mse: 3.5364e-05\n",
      "Epoch 457/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 3.4690e-05 - mse: 3.4690e-05\n",
      "Epoch 458/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 3.4027e-05 - mse: 3.4027e-05\n",
      "Epoch 459/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 3.3378e-05 - mse: 3.3378e-05\n",
      "Epoch 460/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 3.2743e-05 - mse: 3.2743e-05\n",
      "Epoch 461/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 3.2116e-05 - mse: 3.2116e-05\n",
      "Epoch 462/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 3.1504e-05 - mse: 3.1504e-05\n",
      "Epoch 463/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 3.0903e-05 - mse: 3.0903e-05\n",
      "Epoch 464/500\n",
      "1/1 [==============================] - 0s 4ms/step - loss: 3.0313e-05 - mse: 3.0313e-05\n",
      "Epoch 465/500\n",
      "1/1 [==============================] - 0s 19ms/step - loss: 2.9735e-05 - mse: 2.9735e-05\n",
      "Epoch 466/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 2.9168e-05 - mse: 2.9168e-05\n",
      "Epoch 467/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 2.8612e-05 - mse: 2.8612e-05\n",
      "Epoch 468/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 2.8067e-05 - mse: 2.8067e-05\n",
      "Epoch 469/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 2.7531e-05 - mse: 2.7531e-05\n",
      "Epoch 470/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 2.7006e-05 - mse: 2.7006e-05\n",
      "Epoch 471/500\n",
      "1/1 [==============================] - 0s 15ms/step - loss: 2.6490e-05 - mse: 2.6490e-05\n",
      "Epoch 472/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 2.5985e-05 - mse: 2.5985e-05\n",
      "Epoch 473/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 2.5489e-05 - mse: 2.5489e-05\n",
      "Epoch 474/500\n",
      "1/1 [==============================] - 0s 8ms/step - loss: 2.5003e-05 - mse: 2.5003e-05\n",
      "Epoch 475/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 2.4525e-05 - mse: 2.4525e-05\n",
      "Epoch 476/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 2.4058e-05 - mse: 2.4058e-05\n",
      "Epoch 477/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 2.3599e-05 - mse: 2.3599e-05\n",
      "Epoch 478/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 2.3148e-05 - mse: 2.3148e-05\n",
      "Epoch 479/500\n",
      "1/1 [==============================] - 0s 19ms/step - loss: 2.2707e-05 - mse: 2.2707e-05\n",
      "Epoch 480/500\n",
      "1/1 [==============================] - 0s 41ms/step - loss: 2.2274e-05 - mse: 2.2274e-05\n",
      "Epoch 481/500\n",
      "1/1 [==============================] - 0s 13ms/step - loss: 2.1850e-05 - mse: 2.1850e-05\n",
      "Epoch 482/500\n",
      "1/1 [==============================] - 0s 48ms/step - loss: 2.1433e-05 - mse: 2.1433e-05\n",
      "Epoch 483/500\n",
      "1/1 [==============================] - 0s 35ms/step - loss: 2.1024e-05 - mse: 2.1024e-05\n",
      "Epoch 484/500\n",
      "1/1 [==============================] - 0s 12ms/step - loss: 2.0623e-05 - mse: 2.0623e-05\n",
      "Epoch 485/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 2.0229e-05 - mse: 2.0229e-05\n",
      "Epoch 486/500\n",
      "1/1 [==============================] - 0s 9ms/step - loss: 1.9843e-05 - mse: 1.9843e-05\n",
      "Epoch 487/500\n",
      "1/1 [==============================] - 0s 13ms/step - loss: 1.9464e-05 - mse: 1.9464e-05\n",
      "Epoch 488/500\n",
      "1/1 [==============================] - 0s 15ms/step - loss: 1.9094e-05 - mse: 1.9094e-05\n",
      "Epoch 489/500\n",
      "1/1 [==============================] - 0s 22ms/step - loss: 1.8730e-05 - mse: 1.8730e-05\n",
      "Epoch 490/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 1.8372e-05 - mse: 1.8372e-05\n",
      "Epoch 491/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 1.8022e-05 - mse: 1.8022e-05\n",
      "Epoch 492/500\n",
      "1/1 [==============================] - 0s 13ms/step - loss: 1.7678e-05 - mse: 1.7678e-05\n",
      "Epoch 493/500\n",
      "1/1 [==============================] - 0s 28ms/step - loss: 1.7342e-05 - mse: 1.7342e-05\n",
      "Epoch 494/500\n",
      "1/1 [==============================] - 0s 30ms/step - loss: 1.7011e-05 - mse: 1.7011e-05\n",
      "Epoch 495/500\n",
      "1/1 [==============================] - 0s 6ms/step - loss: 1.6685e-05 - mse: 1.6685e-05\n",
      "Epoch 496/500\n",
      "1/1 [==============================] - 0s 11ms/step - loss: 1.6367e-05 - mse: 1.6367e-05\n",
      "Epoch 497/500\n",
      "1/1 [==============================] - 0s 5ms/step - loss: 1.6055e-05 - mse: 1.6055e-05\n",
      "Epoch 498/500\n",
      "1/1 [==============================] - 0s 7ms/step - loss: 1.5749e-05 - mse: 1.5749e-05\n",
      "Epoch 499/500\n",
      "1/1 [==============================] - 0s 3ms/step - loss: 1.5448e-05 - mse: 1.5448e-05\n",
      "Epoch 500/500\n",
      "1/1 [==============================] - 0s 15ms/step - loss: 1.5154e-05 - mse: 1.5154e-05\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(X,y, epochs=500)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ELcrfwfwARF6",
   "metadata": {
    "id": "ELcrfwfwARF6"
   },
   "source": [
    "As you can see, the loss went from `1.2677e-05` all the way down to `8.5635e-10`. If you run it again, these values may change. There are so many randomness involved in neural network training. Take an example, weights initialization is random. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8-GLhw7OA9_M",
   "metadata": {
    "id": "8-GLhw7OA9_M"
   },
   "source": [
    "<a name='2-5'></a>\n",
    "### 2.5  Evaluating a Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bBZxei-vzHAz",
   "metadata": {
    "id": "bBZxei-vzHAz"
   },
   "source": [
    "After we have trained the model, the next step is to evaluate it. \n",
    "\n",
    "But first off, we can plot the loss versus the epochs to see how it performed. Plotting the model metrics is a fundamental step in performing the error analysis. \n",
    "\n",
    "The training metric `mse` and `loss` are contained in `history.history` and the number of epochs are in `history.epoch`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "pSGWj1bK5jDN",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 338
    },
    "id": "pSGWj1bK5jDN",
    "outputId": "e85964dc-cabc-46b7-9753-a5eab0fd2aa3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f8c95ef28d0>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAAEwCAYAAAB14D/FAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAbZklEQVR4nO3dbbCcZZ3n8e//JIEjJgxPITIJTqLGsoBoYBMGKxodqR0enBUspyhZdwiUGl+ghcWsK+gLnClR14ziUMtqMWMkrKLgiGt2ZSXIUEaqFAhMkPDgEJlQJEZyiCJEJkjO+e+Lvs9JA8Fzcrqv9KGv76fq2N1X993973OXh1+upzsyE0mSJHVuoNcFSJIk9QuDlSRJUpcYrCRJkrrEYCVJktQlBitJkqQuMVhJkiR1ybjBKiKOjYjbIuKBiLg/Ii5q2j8VEdsiYmPzc2bbMZdGxOaI+HlEnFbyC0iSJE0VMd4+VhFxDHBMZt4TEbOAu4GzgXOAXZn5dy94/XHAN4GTgT8Gfgi8PjOHX+ozjjrqqJw/f34n30OSJOmAuPvuu5/IzNn7em76eAdn5nZge3P/6Yh4EJj7Bw45C/hWZj4L/FtEbKYVsn7yUgfMnz+fDRs2jFeKJElSz0XEoy/13H7NsYqI+cCJwB1N04cj4mcRsToiDm/a5gKPtR22lX0EsYhYGREbImLD0NDQ/pQhSZI0JU04WEXETOA7wEcz8yngy8BrgcW0erS+sD8fnJlXZ+aSzFwye/Y+e9MkSZJeViYUrCJiBq1Q9Y3MvBEgMx/PzOHMHAH+gdZwH8A24Ni2w+c1bZIkSX1t3DlWERHAV4EHM/OLbe3HNPOvAN4NbGrurwWui4gv0pq8vhC4s6tVS5KknnvuuefYunUru3fv7nUpRQwODjJv3jxmzJgx4WPGDVbAMuCvgPsiYmPT9gng3IhYDCSwBfgQQGbeHxE3AA8Ae4AL/9CKQEmS9PK0detWZs2axfz582n1w/SPzGTnzp1s3bqVBQsWTPi4iawKvB3Y12/rpj9wzOXA5ROuQpIkvezs3r27L0MVQERw5JFHsr8L7Nx5XZIkTVo/hqpRk/luBitJkqQuMVhJkqSXrZkzZ/a6hOepIlj9Zmg7d3z7C/xyy897XYokSepjVQSrnb98hD+9/295/F/v6nUpkiSpgMzkYx/7GCeccAKLFi3i+uuvB2D79u0sX76cxYsXc8IJJ/DjH/+Y4eFhzj///LHXXnHFFV2rYyLbLbzsjU4+G++C05IkaXL+5v/czwO/fKqr73ncHx/KZf/p+Am99sYbb2Tjxo3ce++9PPHEEyxdupTly5dz3XXXcdppp/HJT36S4eFhnnnmGTZu3Mi2bdvYtKm1BeeTTz7ZtZqr6LEiWl8zGOlxIZIkqYTbb7+dc889l2nTpjFnzhze9ra3cdddd7F06VK+9rWv8alPfYr77ruPWbNm8ZrXvIZHHnmEj3zkI/zgBz/g0EMP7VodVfRYDQzYYyVJUkkT7Vk60JYvX8769ev5/ve/z/nnn8/FF1/Meeedx7333svNN9/MV77yFW644QZWr17dlc+rqscqRwxWkiT1o7e+9a1cf/31DA8PMzQ0xPr16zn55JN59NFHmTNnDh/84Af5wAc+wD333MMTTzzByMgI73nPe/j0pz/NPffc07U6quix2rvBl0OBkiT1o3e/+9385Cc/4U1vehMRwec//3le9apXsWbNGlatWsWMGTOYOXMm1157Ldu2beOCCy5gZKSVCz772c92rY46gtXoFXkcCpQkqa/s2rULaHWirFq1ilWrVj3v+RUrVrBixYoXHdfNXqp2lQwFNnOsMFhJkqRyKglWzde0x0qSJBVURbAaXRVosJIkSSVVEaz29lg5eV2SJJVTRbAanbzuPlaSJKmkOoKVQ4GSJOkAqCJY7f2aBitJklROFcHKizBLkqQDoYpgxdjO6wYrSZL6yZYtW3jDG97A+eefz+tf/3re97738cMf/pBly5axcOFC7rzzTn70ox+xePFiFi9ezIknnsjTTz8NwKpVq1i6dClvfOMbueyyy7pSTxU7rw8MtPJj2GMlSVIZ/+8S+NV93X3PVy2CMz437ss2b97Mt7/9bVavXs3SpUu57rrruP3221m7di2f+cxnGB4e5qqrrmLZsmXs2rWLwcFB1q1bx8MPP8ydd95JZvKud72L9evXs3z58o5KrqrHKt1uQZKkvrNgwQIWLVrEwMAAxx9/PKeeeioRwaJFi9iyZQvLli3j4osv5sorr+TJJ59k+vTprFu3jnXr1nHiiSdy0kkn8dBDD/Hwww93XEsVPVZjF2G2x0qSpDIm0LNUysEHHzx2f2BgYOzxwMAAe/bs4ZJLLuGd73wnN910E8uWLePmm28mM7n00kv50Ic+1NVaquiximaDUCevS5JUn1/84hcsWrSIj3/84yxdupSHHnqI0047jdWrV49dxHnbtm3s2LGj48+qq8fKyeuSJFXnS1/6ErfddtvYUOEZZ5zBwQcfzIMPPsib3/xmAGbOnMnXv/51jj766I4+q45ghUOBkiT1o/nz57Np06axx9dcc81LPvdCF110ERdddFFX66liKJABNwiVJEnlVRGs9k5ed1WgJEkqp7JgZY+VJEnd1M8Lwybz3SoJVq4KlCSp2wYHB9m5c2df/vc1M9m5cyeDg4P7dVwdk9ddFShJUtfNmzePrVu3MjQ01OtSihgcHGTevHn7dUxdwaoPE7UkSb0yY8YMFixY0OsyppQqhgIJVwVKkqTyqghWMbrdgqsCJUlSQXUEK4cCJUnSAVBXsHIoUJIkFVRXsLLHSpIkFWSwkiRJ6pJKglWzQWiP65AkSf2tjmDlqkBJknQA1BGsmqHAcChQkiQVVEmwGh0KtMdKkiSVU0mwcvK6JEkqb9xgFRHHRsRtEfFARNwfERc17UdExC0R8XBze3jTHhFxZURsjoifRcRJpb/EePbuYyVJklTORHqs9gB/nZnHAacAF0bEccAlwK2ZuRC4tXkMcAawsPlZCXy561Xvp9GhQHusJElSSeMGq8zcnpn3NPefBh4E5gJnAWual60Bzm7unwVcmy0/BQ6LiGO6Xvl+GBjwIsySJKm8/ZpjFRHzgROBO4A5mbm9eepXwJzm/lzgsbbDtjZtL3yvlRGxISI2DA0N7WfZ+8ftFiRJ0oEw4WAVETOB7wAfzcyn2p/LzGQ/u4My8+rMXJKZS2bPnr0/h06a2y1IkqSSJhSsImIGrVD1jcy8sWl+fHSIr7nd0bRvA45tO3xe09ZTIxmkQ4GSJKmgiawKDOCrwIOZ+cW2p9YCK5r7K4DvtbWf16wOPAX4bduQYc8kOHldkiQVNX0Cr1kG/BVwX0RsbNo+AXwOuCEi3g88CpzTPHcTcCawGXgGuKCrFU9SEjh5XZIklTRusMrM24GX2gjq1H28PoELO6yr65Kwx0qSJBVVxc7rYI+VJEkqr65g5XYLkiSpoIqCldstSJKksioKVl4vUJIklVVXsHIoUJIkFVRNsJIkSSqtmmA1wgCuCpQkSSVVE6xaO687FChJksqpJ1iFG4RKkqSyqglWEIRDgZIkqaBqgpUXYZYkSaVVFKy8pI0kSSqromBVzVeVJEk9Uk3acFWgJEkqraJg5VCgJEkqq65g5eR1SZJUUDXBCnC7BUmSVFQ1wcoeK0mSVFpdwcoeK0mSVFBdwcoeK0mSVFBVwco5VpIkqaRqghX2WEmSpMKqCVbZ9r+SJEkl1BOswsnrkiSprGqCFQThUKAkSSqommA14nYLkiSpsGqClZPXJUlSadUEK7dbkCRJpVUTrFoMVpIkqZxqglWGQ4GSJKmseoKVQ4GSJKmwaoIVrgqUJEmFVROsvAizJEkqrapg5VCgJEkqqZpg5T5WkiSptGqClRdhliRJpdUTrCJ6XYIkSepz1QQrnGMlSZIKqyZYuSpQkiSVVlWwCkZ6XYYkSepj1QQrvKSNJEkqrJpg1eqxkiRJKmfcYBURqyNiR0Rsamv7VERsi4iNzc+Zbc9dGhGbI+LnEXFaqcInxx4rSZJUzkR6rK4BTt9H+xWZubj5uQkgIo4D3gsc3xzzPyNiWreK7UR6rUBJklTYuMEqM9cDv57g+50FfCszn83MfwM2Ayd3UF/XJAOEc6wkSVJBncyx+nBE/KwZKjy8aZsLPNb2mq1N24tExMqI2BARG4aGhjooY4LCVYGSJKmsyQarLwOvBRYD24Ev7O8bZObVmbkkM5fMnj17kmXsx+cV/wRJklS7SQWrzHw8M4czcwT4B/YO920Djm176bymbQoIhwIlSVJRkwpWEXFM28N3A6MrBtcC742IgyNiAbAQuLOzErvDyeuSJKm06eO9ICK+CbwdOCoitgKXAW+PiMW0ksoW4EMAmXl/RNwAPADsAS7MzOEype+nMFhJkqSyxg1WmXnuPpq/+gdefzlweSdFlZDgBqGSJKmoanZehwEiXRUoSZLKqSZYpUOBkiSpsGqCFTgUKEmSyqomWCUBbrcgSZIKqiZYQRAOBUqSpIKqCVbOsZIkSaVVE6xgwB4rSZJUVDXBqrWPlcFKkiSVU02wIpy8LkmSyqonWDl5XZIkFVZNsEp3sZIkSYVVE6wIe6wkSVJZ1QSrVo+VwUqSJJVTTbCCYMDJ65IkqaB6gpUbhEqSpMKqCVbpqkBJklRYNcEK51hJkqTC6glWgRsuSJKkouoJVvZYSZKkwqoJVhlehFmSJJVVTbCCINxuQZIkFVRNsHJVoCRJKq2aYOU+VpIkqbR6ghXYYyVJkoqqKFiF2y1IkqSiqglWGQM4FChJkkqqJlh5EWZJklRaPcHKyeuSJKmwaoKV2y1IkqTSqglW4KpASZJUVj3BKlwTKEmSyqonWDkUKEmSCqsmWHkRZkmSVFo1wYqwx0qSJJVVT7ByKFCSJBVWUbDCS9pIkqSi6glWXtJGkiQVVk+wwn2sJElSWfUEK1cFSpKkwqoJVl7SRpIklVZNsCKCSIOVJEkqp55gRbgqUJIkFVVPsHKDUEmSVNi4wSoiVkfEjojY1NZ2RETcEhEPN7eHN+0REVdGxOaI+FlEnFSy+P1jsJIkSWVNpMfqGuD0F7RdAtyamQuBW5vHAGcAC5uflcCXu1NmF0Q9nXOSJKk3xk0bmbke+PULms8C1jT31wBnt7Vfmy0/BQ6LiGO6VWynBhjpdQmSJKmPTbYbZ05mbm/u/wqY09yfCzzW9rqtTduLRMTKiNgQERuGhoYmWcbEZTh5XZIkldXx+FhmJpO4VkxmXp2ZSzJzyezZszstYwLcIFSSJJU12WD1+OgQX3O7o2nfBhzb9rp5TVvvBXitQEmSVNJkg9VaYEVzfwXwvbb285rVgacAv20bMuwxhwIlSVJZ08d7QUR8E3g7cFREbAUuAz4H3BAR7wceBc5pXn4TcCawGXgGuKBAzZPjPlaSJKmwcYNVZp77Ek+duo/XJnBhp0UVEQMMGKwkSVJB1WzuFAQDYbCSJEnlVBOsMpxhJUmSyqomWDXLAskRNwmVJEll1BOsmh6r1jQwSZKk7qsnWGGwkiRJZdUTrMZ6rBwKlCRJZVQXrEacYyVJkgqpJlhF81XtsZIkSaVUE6yy2W3BOVaSJKmUaoLV6OR1DFaSJKmQeoJVjA4FGqwkSVIZ1QSrcFWgJEkqrJpg5apASZJUWj3Byg1CJUlSYfUEKy9pI0mSCqsnWNljJUmSCqsnWIXbLUiSpLLqCVZj+1g5eV2SJJVRT7ByHytJklRYNcEqxrZbGO5xJZIkqV9VE6xcFShJkkqrJ1i5KlCSJBVWT7AaXRWIwUqSJJVRT7AaXRU4YrCSJEll1BOsRlcF2mMlSZIKqSZYuSpQkiSVVk2wclWgJEkqrZ5ghZe0kSRJZVUTrGKsx8pL2kiSpDKqCVY5Fqx6XIgkSepb1QQrhwIlSVJp1QSrGLsIs6sCJUlSGdUEq7FVgW4QKkmSCqkmWI1NXneDUEmSVEg1wWrvHCtXBUqSpDLqCVZuECpJkgqrMFj1uA5JktS3qglWo6sCHQqUJEmlVBOscOd1SZJUWD3BitF9rBwLlCRJZVQTrJoOK4cCJUlSMdUEKyevS5Kk0qZ3cnBEbAGeBoaBPZm5JCKOAK4H5gNbgHMy8zedldm5cLsFSZJUWDd6rP4sMxdn5pLm8SXArZm5ELi1edx7Ma1161CgJEkqpMRQ4FnAmub+GuDsAp8xaa4KlCRJpXQarBJYFxF3R8TKpm1OZm5v7v8KmLOvAyNiZURsiIgNQ0NDHZYxvtF9rBwKlCRJpXQ0xwp4S2Zui4ijgVsi4qH2JzMzI2KfSSYzrwauBliyZEn5tDO6LNBgJUmSCumoxyoztzW3O4DvAicDj0fEMQDN7Y5Oi+wKJ69LkqTCJh2sIuKVETFr9D7w58AmYC2wonnZCuB7nRbZDWP7WGGwkiRJZXQyFDgH+G6zjcF04LrM/EFE3AXcEBHvBx4Fzum8zG5o5liNOHldkiSVMelglZmPAG/aR/tO4NROiiph7+R1g5UkSSqjup3XnbwuSZJKqSZYhcFKkiQVVk2wGlsV6OR1SZJUSD3BCjcIlSRJZVUTrMa2WzBYSZKkQqoJVgzYYyVJksqqJljF2FDgcI8rkSRJ/aqaYOV2C5IkqbRqgpXbLUiSpNKqCVZ7L8Lc4zokSVLfqiZYjfVY4SVtJElSGRUFq+ar2mUlSZIKqSZYjQ0FjthjJUmSyqgvWHlJG0mSVEg1wSpwVaAkSSqrnmA1MPpVHQqUJEllVBOswO0WJElSWdUEKzcIlSRJpdUTrAbcbkGSJJVVTbDaOxToHCtJklRGNcEqxi5pY4+VJEkqo5pgRbPzergqUJIkFVJNsLLHSpIklVZRsGruGKwkSVIh1QSr6QcNAjDy3LM9rkSSJPWraoLVK155GADDu5/ucSWSJKlfVROsDjm0Faxy91M9rkSSJPWraoLVKw6ZxXAG+ftdvS5FkiT1qWqCVQwM8AyDxLMOBUqSpDKqCVYAz8QhDDz3u16XIUmS+lRVwWr3wCFM3+NQoCRJKqPCYGWPlSRJKqOqYPX7aYdw0J5nel2GJEnqU1UFq+emz2RwxB4rSZJURlXBanj6Kxkc+fdelyFJkvpUVcFqZMYrOQSHAiVJUhl1BauDZ3FI/js5MtLrUiRJUh+qKlhx0EymxwjP7rbXSpIkdV9VwWpg8FAAdj31mx5XIkmS+lFlwWoWALt3/bbHlUiSpH5UVbCa/opWj9Xu3z3Z40okSVI/qipYzTikFaye/d1TPa5EkiT1o2LBKiJOj4ifR8TmiLik1Ofsj4MO+SMAnnvGoUBJktR9RYJVREwDrgLOAI4Dzo2I40p81v44dPZcAHZv/nGPK5EkSf1oeqH3PRnYnJmPAETEt4CzgAcKfd6EvOrY13HnYWfyH7Z/izuuepaR6YdAROuHIGKAbO632ttyZ8Teu00ezdjb0roZaH+095ixY/e+x4vaxnkcHbzXSx0bbd8pX/K55x/7Uu2T0qNjgx7V3MG/Yzr62E5+Vx0cmx19306+8GQ/s6qZEVLfOvSY1/KaE/60Z59fKljNBR5re7wVeN63jIiVwEqAV7/61YXKeLHX/ee/41//cSvH7/g+B/EcrT//SZAEMBB5wGqRJEnddceRZ/dlsBpXZl4NXA2wZMmSA5Zmjjh6Lkd84g8PBebICJnJyMhw63HuLW/0fubI858ba9/38/t+j9zne/ASx77oM593v7kdaV7LOMc+7z1Gnt/0opqb51/w3qN1Tkb7d9j/Yyd9aEc1d/LBY7+zSX1sB1+4o3PUwcf2oOaOPrOD8yNpavmTw+f09PNLBattwLFtj+c1bS8LMTDQ6r2aNq3XpUiSpJeRUpMK7gIWRsSCiDgIeC+wttBnSZIkTQlFeqwyc09EfBi4GZgGrM7M+0t8liRJ0lRRbI5VZt4E3FTq/SVJkqYa1xdLkiR1icFKkiSpSwxWkiRJXWKwkiRJ6hKDlSRJUpcYrCRJkrrEYCVJktQl0dn1tbpURMQQ8OgB+KijgCcOwOdo4jwnU5PnZWryvEw9npOpqfR5+ZPMnL2vJ6ZEsDpQImJDZi7pdR3ay3MyNXlepibPy9TjOZmaenleHAqUJEnqEoOVJElSl9QWrK7udQF6Ec/J1OR5mZo8L1OP52Rq6tl5qWqOlSRJUkm19VhJkiQVY7CSJEnqkiqCVUScHhE/j4jNEXFJr+upSUSsjogdEbGpre2IiLglIh5ubg9v2iMirmzO088i4qTeVd6/IuLYiLgtIh6IiPsj4qKm3fPSQxExGBF3RsS9zXn5m6Z9QUTc0fz+r4+Ig5r2g5vHm5vn5/ey/n4WEdMi4l8i4v82jz0nPRYRWyLivojYGBEbmrYp8Tes74NVREwDrgLOAI4Dzo2I43pbVVWuAU5/QdslwK2ZuRC4tXkMrXO0sPlZCXz5ANVYmz3AX2fmccApwIXN/yc8L731LPCOzHwTsBg4PSJOAf47cEVmvg74DfD+5vXvB37TtF/RvE5lXAQ82PbYczI1/FlmLm7br2pK/A3r+2AFnAxszsxHMvP3wLeAs3pcUzUycz3w6xc0nwWsae6vAc5ua782W34KHBYRxxyYSuuRmdsz857m/tO0/oMxF89LTzW/313NwxnNTwLvAP6paX/heRk9X/8EnBoRcYDKrUZEzAPeCfxj8zjwnExVU+JvWA3Bai7wWNvjrU2bemdOZm5v7v8KmNPc91wdYM1QxYnAHXheeq4ZctoI7ABuAX4BPJmZe5qXtP/ux85L8/xvgSMPbMVV+BLw34CR5vGReE6mggTWRcTdEbGyaZsSf8Oml3pjaSIyMyPCPT96ICJmAt8BPpqZT7X/w9rz0huZOQwsjojDgO8Cb+hxSVWLiL8AdmTm3RHx9l7Xo+d5S2Zui4ijgVsi4qH2J3v5N6yGHqttwLFtj+c1beqdx0e7YZvbHU275+oAiYgZtELVNzLzxqbZ8zJFZOaTwG3Am2kNW4z+I7j9dz92Xprn/wjYeYBL7XfLgHdFxBZa00jeAfw9npOey8xtze0OWv8IOZkp8jeshmB1F7CwWcVxEPBeYG2Pa6rdWmBFc38F8L229vOaFRynAL9t69ZVlzRzPr4KPJiZX2x7yvPSQxExu+mpIiJeAfxHWvPfbgP+snnZC8/L6Pn6S+Cf0x2fuyozL83MeZk5n9Z/O/45M9+H56SnIuKVETFr9D7w58AmpsjfsCp2Xo+IM2mNk08DVmfm5T0uqRoR8U3g7cBRwOPAZcD/Bm4AXg08CpyTmb9u/oP/P2itInwGuCAzN/Si7n4WEW8Bfgzcx955I5+gNc/K89IjEfFGWhNup9H6R+8Nmfm3EfEaWr0lRwD/AvyXzHw2IgaB/0Vrjtyvgfdm5iO9qb7/NUOB/zUz/8Jz0lvN7/+7zcPpwHWZeXlEHMkU+BtWRbCSJEk6EGoYCpQkSTogDFaSJEldYrCSJEnqEoOVJElSlxisJEmSusRgJUmS1CUGK0mSpC75/y80RM9YFTguAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss_df = pd.DataFrame(history.history)\n",
    "\n",
    "# Plot loss vs epochs\n",
    "\n",
    "loss_df.plot(figsize=(10,5))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "VolEuUKs-LvT",
   "metadata": {
    "id": "VolEuUKs-LvT"
   },
   "source": [
    "There are cowple of things to note from the above graph. \n",
    "\n",
    "* First off, the loss and metric we are tracking are all similar. They are both `mean_squared_error` or `mse`.\n",
    "* The model had no improvement from 80(approx) epochs. This means that 500 epochs was too much, and instead of burning resources or compute power, we would have trained for fewer epochs since the model does not show a significant improvements in the later epochs. \n",
    "\n",
    "Training for many epochs beyond what's needed is usually the cause of `overfitting`. We will learn more about overfitting later, but simply, it's when the model is so good on the training set but very poor on the test set or the new data. So, in our case we are forcing the model to fit the data by training it too much. Overfitting can also be caused by using bigger models (for small dataset), etc...\n",
    "\n",
    "\n",
    "That said, let's make some predictions on unseen data. For simplicity, let's predict `y` of `X=30`. Remember that our equation is `y=2X+1`, so with x=30, y should be equal to `61`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "u1oA77b0__LG",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "u1oA77b0__LG",
    "outputId": "38d7149b-0743-4dba-a23c-5fc81ed06272"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[61.014885]], dtype=float32)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict([30.0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "iu2KiU4yA6pP",
   "metadata": {
    "id": "iu2KiU4yA6pP"
   },
   "source": [
    "Wow! That's so impressive. \n",
    "\n",
    "The model was able to determine the relationship between `X` and `y` and can use that relationship to predict `y` for unseen values of `X`. \n",
    "\n",
    "One thing to note that it is not guarranted to get the exact predictions, say 61. This is because there are so many randomness and probabilities involved behind the scene. \n",
    "\n",
    "\n",
    "One last thing we can try is to get the model parameters, that is weight and bias. And their values should be close to the coeffient and intercept of our linear equation, y=2X+1. 2 is coefficient(or weight), and 1 is the intercept(or bias). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b29uAewjCjB9",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "b29uAewjCjB9",
    "outputId": "35ab71a7-b1ac-4418-8665-742a3c0a3aa1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([[2.0006804]], dtype=float32), array([0.994472], dtype=float32)]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Getting the model weights\n",
    "\n",
    "model.get_weights()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "TR61EgS-C-yl",
   "metadata": {
    "id": "TR61EgS-C-yl"
   },
   "source": [
    "So, as you can see, the model learned that the relationship between X and y is `y=2.0008771 + 0.9928744` and this is very close to `y=2X+1`. Something intringuing here is that there is no where told the model such relationship - `it simply learned it observing the data that we provided`, and `this is the basis of the idea that machine learning is used to extract patterns in data`. \n",
    "\n",
    "With complex data, you might not get such intuition because there will be so many parameters, but in our case, the model was simple, the data was simple, and that allowed us to `uncover` the principal idea behind machine learning - `learning the relationship/pattern between the input features and labels, and using such relationship to make predictions on unseen data.`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "SjI67-vME-2y",
   "metadata": {
    "id": "SjI67-vME-2y"
   },
   "source": [
    "<a name='2-6'></a>\n",
    "### 2.6  Improving the results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "FJNcx7oRFC-P",
   "metadata": {
    "id": "FJNcx7oRFC-P"
   },
   "source": [
    "Ideally, your model will likely not be good at the first. \n",
    "\n",
    "You will need to tweak some hyperparameters, or even improve the data. Also, there is a notion that [machine learning model is only 5% of what are to be done to ship a working machine learning system](https://jeande.medium.com/ml-model-is-5-what-should-we-be-doing-cd68ae14ad7f).  So, often, all you need is to improve the data than improving the model. There are even state of the art and open source models that you can take an advantages of if you have good data. And in those intances, building model will be out of the equation.  \n",
    "\n",
    "But of course, improving the model and performing error analysis is not a trivial task and will depend on the results of the model on training and testing set. Here is some ideas that can guide you: \n",
    "\n",
    "* If the model is not doing well on the training data, it's a clue that the input data (X) doesn't contain the useful information needed to predict the output y. Or put it simply, the input features do not have `high predictive power`. The right thing to do here is to improve the data. Otherwise, the problem will perssit. \n",
    "\n",
    "* If the model is doing well on the training data but poorly on the testing data, it maybe that you overfitted the training data and that resulted in model failing to generalize on test/new data. Overfitting is one thing, there maybe other things not going well or worth improving. The right thing to do here is to plot the learning curve and see what's to be done based off what you are seeing.\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eftCBSkUSCYV",
   "metadata": {
    "id": "eftCBSkUSCYV"
   },
   "source": [
    "Up to now, we have come a long way doing regression with neural networks. We have learned how to create a simple data, how to create, train, and compile a simple model, evaluating the results, and we saw some ideas on perfoming error analysis. \n",
    "\n",
    "We started simple with the goal of getting prepared to take a step further into real world scanerios. I wanted to jump quicky to bigger models and computer vision things but I remembered that quite often, it is understanding the basics that can set us off for understanding the bigger picture. \n",
    "\n",
    "To make it more exciting, let's not stop on linear equation (we could after all, we did regression already), but let's step into real world dataset, still practicing regression. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Ce1AurnvUWUr",
   "metadata": {
    "id": "Ce1AurnvUWUr"
   },
   "source": [
    "<a name='3'></a>\n",
    "\n",
    "## 3. Going Beyond: A Real world dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "MbhDb1hgWjy_",
   "metadata": {
    "id": "MbhDb1hgWjy_"
   },
   "source": [
    "Welome to the second part of the notebook, where we leap into real world scenarios. \n",
    "\n",
    "Still doing regression, we will use the real world forest dataset to predict the burned area of forest fires, in the northeast region of Portugal, by using meteorological and other data. \n",
    "\n",
    "You can learn more about the dataset [here](https://archive.ics.uci.edu/ml/datasets/Forest+Fires)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "_RplpeLNUYj0",
   "metadata": {
    "id": "_RplpeLNUYj0"
   },
   "source": [
    "<a name='3-1'></a>\n",
    "\n",
    "## 3.1 Loading the data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "YFVMmxtm6R_a",
   "metadata": {
    "id": "YFVMmxtm6R_a"
   },
   "source": [
    "Before loading the dataset, I will first import all relevant imports. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "JkG09wMBNVpf",
   "metadata": {
    "id": "JkG09wMBNVpf"
   },
   "source": [
    "Here are the information about the attributes: \n",
    "\n",
    "1. X - x-axis spatial coordinate within the Montesinho park map: 1 to 9 \n",
    "2. Y - y-axis spatial coordinate within the Montesinho park map: 2 to 9 \n",
    "3. month - month of the year: 'jan' to 'dec' \n",
    "4. day - day of the week: 'mon' to 'sun' \n",
    "5. FFMC - FFMC index from the FWI system: 18.7 to 96.20 \n",
    "6. DMC - DMC index from the FWI system: 1.1 to 291.3 \n",
    "7. DC - DC index from the FWI system: 7.9 to 860.6 \n",
    "8. ISI - ISI index from the FWI system: 0.0 to 56.10 \n",
    "9. temp - temperature in Celsius degrees: 2.2 to 33.30 \n",
    "10. RH - relative humidity in %: 15.0 to 100 \n",
    "11. wind - wind speed in km/h: 0.40 to 9.40 \n",
    "12. rain - outside rain in mm/m2 : 0.0 to 6.4 \n",
    "13. area - the burned area of the forest (in ha): 0.00 to 1090.84 \n",
    "\n",
    "Source: https://archive.ics.uci.edu/ml/datasets/Forest+**Fires\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "t4HzO0qP6ZyU",
   "metadata": {
    "id": "t4HzO0qP6ZyU"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "BWPs93SeGNrf",
   "metadata": {
    "id": "BWPs93SeGNrf"
   },
   "source": [
    "Let's download the dataset and load it into a Pandas dataframe using `pd.read_csv()`,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "k6fog1WK9-bt",
   "metadata": {
    "id": "k6fog1WK9-bt"
   },
   "outputs": [],
   "source": [
    "dataset_url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/forest-fires/forestfires.csv'\n",
    "\n",
    "forest_df = pd.read_csv(dataset_url)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "mtdI1_o1G_6g",
   "metadata": {
    "id": "mtdI1_o1G_6g"
   },
   "source": [
    "Let's see the features and their data types"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "lZyX4Wg1G4ac",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lZyX4Wg1G4ac",
    "outputId": "8ef2c474-ee50-42e3-8296-c15e24cb26ca"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 517 entries, 0 to 516\n",
      "Data columns (total 13 columns):\n",
      " #   Column  Non-Null Count  Dtype  \n",
      "---  ------  --------------  -----  \n",
      " 0   X       517 non-null    int64  \n",
      " 1   Y       517 non-null    int64  \n",
      " 2   month   517 non-null    object \n",
      " 3   day     517 non-null    object \n",
      " 4   FFMC    517 non-null    float64\n",
      " 5   DMC     517 non-null    float64\n",
      " 6   DC      517 non-null    float64\n",
      " 7   ISI     517 non-null    float64\n",
      " 8   temp    517 non-null    float64\n",
      " 9   RH      517 non-null    int64  \n",
      " 10  wind    517 non-null    float64\n",
      " 11  rain    517 non-null    float64\n",
      " 12  area    517 non-null    float64\n",
      "dtypes: float64(8), int64(3), object(2)\n",
      "memory usage: 52.6+ KB\n"
     ]
    }
   ],
   "source": [
    "forest_df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "mNrE3bEyJtP9",
   "metadata": {
    "id": "mNrE3bEyJtP9"
   },
   "source": [
    "The dataset contains 517 examples and 13 columns, 12 features and 1 label (`areas`). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "ufgJHWdhG-Rm",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ufgJHWdhG-Rm",
    "outputId": "8343fd06-75a4-4a87-a072-909be25601e5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(517, 13)\n"
     ]
    }
   ],
   "source": [
    "print(forest_df.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sxFGkjy8KqvN",
   "metadata": {
    "id": "sxFGkjy8KqvN"
   },
   "source": [
    "<a name='3-2'></a>\n",
    "\n",
    "## 3.2 Looking in the data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "tF_IKN-JKyYI",
   "metadata": {
    "id": "tF_IKN-JKyYI"
   },
   "source": [
    "We will not go deep into analysis, but let's try to learn about the data we have. Before that, I will first split the dataset into training and test set. \n",
    "\n",
    "I will use Scikit-Learn `train_test_split`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "dj61Dr0aKtfC",
   "metadata": {
    "id": "dj61Dr0aKtfC"
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "train_data, test_data = train_test_split(forest_df, test_size=0.3, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "e5H1K90wL-I2",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "e5H1K90wL-I2",
    "outputId": "43644c34-1b68-4caa-9015-57d4341cc154"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The shape of training data: (361, 13)\n",
      "The shape of testing data: (156, 13)\n"
     ]
    }
   ],
   "source": [
    "print('The shape of training data: {}\\nThe shape of testing data: {}'.format(train_data.shape, test_data.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "rJLgRbRfLb91",
   "metadata": {
    "id": "rJLgRbRfLb91"
   },
   "source": [
    "Let's peep into the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "9G5OXBW_LYzA",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "id": "9G5OXBW_LYzA",
    "outputId": "3bf727ab-f125-4a31-dde9-e7143a463708"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>month</th>\n",
       "      <th>day</th>\n",
       "      <th>FFMC</th>\n",
       "      <th>DMC</th>\n",
       "      <th>DC</th>\n",
       "      <th>ISI</th>\n",
       "      <th>temp</th>\n",
       "      <th>RH</th>\n",
       "      <th>wind</th>\n",
       "      <th>rain</th>\n",
       "      <th>area</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>311</th>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>sep</td>\n",
       "      <td>sun</td>\n",
       "      <td>92.4</td>\n",
       "      <td>105.8</td>\n",
       "      <td>758.1</td>\n",
       "      <td>9.9</td>\n",
       "      <td>24.8</td>\n",
       "      <td>28</td>\n",
       "      <td>1.8</td>\n",
       "      <td>0.0</td>\n",
       "      <td>14.29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>368</th>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>sep</td>\n",
       "      <td>sat</td>\n",
       "      <td>91.2</td>\n",
       "      <td>94.3</td>\n",
       "      <td>744.4</td>\n",
       "      <td>8.4</td>\n",
       "      <td>16.8</td>\n",
       "      <td>47</td>\n",
       "      <td>4.9</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>aug</td>\n",
       "      <td>sat</td>\n",
       "      <td>90.2</td>\n",
       "      <td>110.9</td>\n",
       "      <td>537.4</td>\n",
       "      <td>6.2</td>\n",
       "      <td>19.5</td>\n",
       "      <td>43</td>\n",
       "      <td>5.8</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>271</th>\n",
       "      <td>8</td>\n",
       "      <td>6</td>\n",
       "      <td>aug</td>\n",
       "      <td>tue</td>\n",
       "      <td>92.1</td>\n",
       "      <td>152.6</td>\n",
       "      <td>658.2</td>\n",
       "      <td>14.3</td>\n",
       "      <td>20.1</td>\n",
       "      <td>58</td>\n",
       "      <td>4.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>299</th>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>jun</td>\n",
       "      <td>sat</td>\n",
       "      <td>53.4</td>\n",
       "      <td>71.0</td>\n",
       "      <td>233.8</td>\n",
       "      <td>0.4</td>\n",
       "      <td>10.6</td>\n",
       "      <td>90</td>\n",
       "      <td>2.7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     X  Y month  day  FFMC    DMC     DC   ISI  temp  RH  wind  rain   area\n",
       "311  6  3   sep  sun  92.4  105.8  758.1   9.9  24.8  28   1.8   0.0  14.29\n",
       "368  6  5   sep  sat  91.2   94.3  744.4   8.4  16.8  47   4.9   0.0  12.64\n",
       "23   7  4   aug  sat  90.2  110.9  537.4   6.2  19.5  43   5.8   0.0   0.00\n",
       "271  8  6   aug  tue  92.1  152.6  658.2  14.3  20.1  58   4.5   0.0   9.27\n",
       "299  6  5   jun  sat  53.4   71.0  233.8   0.4  10.6  90   2.7   0.0   0.00"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "T6j7bE2qLywN",
   "metadata": {
    "id": "T6j7bE2qLywN"
   },
   "source": [
    "It seems that we have two categorical features, `month` and `day`. We will remember to encode them. For now we can see the number of samples in each month and later in each day. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "01Ag_eK4MVxx",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 348
    },
    "id": "01Ag_eK4MVxx",
    "outputId": "f35b5f2c-999d-40b9-9f02-fd15ddde0e7d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f8c8ef60550>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAAE6CAYAAADUexyjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAY50lEQVR4nO3de7RmdX3f8fdHRsU7KicsBMugUikaryPiklqV2GKIQBLipQYJYqlWg4mpEU0rJsYEGy8xtlInImJiVIJYaGxMKfUaFR2QcNU4VdGhKMcoxOIV/faPvY8cxpk5Z87zO2fvfXi/1nLN2b/n9nHWsM/n+e3f83tSVUiSJGl2dxg6gCRJ0nphsZIkSWrEYiVJktSIxUqSJKkRi5UkSVIjG4YOALD33nvXxo0bh44hSZK0pEsuueQbVTW3o9tGUaw2btzIli1bho4hSZK0pCTX7uw2LwVKkiQ1YrGSJElqxGIlSZLUiMVKkiSpEYuVJElSIxYrSZKkRixWkiRJjVisJEmSGrFYSZIkNWKxkiRJasRiJUmS1Mgovitwd2089QOr9txfPv2oVXtuSZK0vjljJUmS1IjFSpIkqRGLlSRJUiOTXGM1Vau5NgxcHyZJ0tCWnLFK8vYkNyS5ctHYHyX5XJLLk7w/yV6Lbnt5kq1JPp/kX61WcEmSpLFZzqXAdwBHbjd2IfDQqnoY8PfAywGSHAI8E3hI/5i3JNmjWVpJkqQRW7JYVdVHgW9uN/Y/q+qW/vBTwP79z8cA76mq71fVl4CtwKEN80qSJI1Wi8XrzwX+uv95P+Cri27b1o9JkiStezMVqyS/A9wCvGsFjz05yZYkW+bn52eJIUmSNAorLlZJfg34BeDZVVX98HXA/Rfdbf9+7KdU1eaq2lRVm+bm5lYaQ5IkaTRWVKySHAn8NnB0VX1n0U0XAM9McuckBwIHAZ+ePaYkSdL4LbmPVZJ3A08E9k6yDTiN7lOAdwYuTALwqap6flVdleQc4Gq6S4QvrKofrVZ4SZKkMVmyWFXVs3YwfOYu7v8a4DWzhJIkSZoiv9JGkiSpEYuVJElSIxYrSZKkRixWkiRJjVisJEmSGrFYSZIkNWKxkiRJasRiJUmS1IjFSpIkqRGLlSRJUiMWK0mSpEYsVpIkSY1YrCRJkhqxWEmSJDVisZIkSWrEYiVJktSIxUqSJKkRi5UkSVIjFitJkqRGLFaSJEmNWKwkSZIasVhJkiQ1YrGSJElqxGIlSZLUiMVKkiSpEYuVJElSIxYrSZKkRixWkiRJjVisJEmSGrFYSZIkNWKxkiRJasRiJUmS1IjFSpIkqZEli1WStye5IcmVi8buk+TCJF/o/7x3P54kf5Jka5LLkzxqNcNLkiSNyXJmrN4BHLnd2KnARVV1EHBRfwzwVOCg/n8nA2e0iSlJkjR+Sxarqvoo8M3tho8Bzu5/Phs4dtH4O6vzKWCvJPu2CitJkjRmK11jtU9VXd///DVgn/7n/YCvLrrftn7spyQ5OcmWJFvm5+dXGEOSJGk8Zl68XlUF1Aoet7mqNlXVprm5uVljSJIkDW6lxerrC5f4+j9v6MevA+6/6H7792OSJEnr3kqL1QXACf3PJwDnLxp/Tv/pwMOAmxZdMpQkSVrXNix1hyTvBp4I7J1kG3AacDpwTpKTgGuBp/d3/x/AzwNbge8AJ65CZkmSpFFaslhV1bN2ctMRO7hvAS+cNZQkSdIUufO6JElSIxYrSZKkRixWkiRJjVisJEmSGrFYSZIkNWKxkiRJasRiJUmS1IjFSpIkqRGLlSRJUiMWK0mSpEYsVpIkSY1YrCRJkhqxWEmSJDVisZIkSWrEYiVJktSIxUqSJKkRi5UkSVIjFitJkqRGLFaSJEmNWKwkSZIasVhJkiQ1YrGSJElqxGIlSZLUiMVKkiSpEYuVJElSIxYrSZKkRixWkiRJjVisJEmSGrFYSZIkNWKxkiRJasRiJUmS1IjFSpIkqRGLlSRJUiMzFaskv5nkqiRXJnl3kj2THJjk4iRbk7w3yZ1ahZUkSRqzFRerJPsBpwCbquqhwB7AM4HXAm+sqgcB3wJOahFUkiRp7Ga9FLgBuEuSDcBdgeuBJwPn9refDRw742tIkiRNwoqLVVVdB7wO+ApdoboJuAS4sapu6e+2DdhvR49PcnKSLUm2zM/PrzSGJEnSaMxyKfDewDHAgcD9gLsBRy738VW1uao2VdWmubm5lcaQJEkajVkuBf4c8KWqmq+qHwLnAY8H9uovDQLsD1w3Y0ZJkqRJmKVYfQU4LMldkwQ4Arga+BBwXH+fE4DzZ4soSZI0DbOssbqYbpH6pcAV/XNtBl4GvCTJVuC+wJkNckqSJI3ehqXvsnNVdRpw2nbDXwQOneV5JUmSpsid1yVJkhqxWEmSJDVisZIkSWrEYiVJktSIxUqSJKkRi5UkSVIjFitJkqRGLFaSJEmNWKwkSZIasVhJkiQ1YrGSJElqxGIlSZLUiMVKkiSpEYuVJElSIxYrSZKkRixWkiRJjVisJEmSGrFYSZIkNWKxkiRJasRiJUmS1IjFSpIkqRGLlSRJUiMWK0mSpEYsVpIkSY1YrCRJkhqxWEmSJDVisZIkSWrEYiVJktSIxUqSJKkRi5UkSVIjFitJkqRGLFaSJEmNWKwkSZIamalYJdkryblJPpfkmiSPS3KfJBcm+UL/571bhZUkSRqzWWes3gR8sKoOBh4OXAOcClxUVQcBF/XHkiRJ696Ki1WSewFPAM4EqKofVNWNwDHA2f3dzgaOnTWkJEnSFMwyY3UgMA+cleSzSd6W5G7APlV1fX+frwH77OjBSU5OsiXJlvn5+RliSJIkjcMsxWoD8CjgjKp6JHAz2132q6oCakcPrqrNVbWpqjbNzc3NEEOSJGkcZilW24BtVXVxf3wuXdH6epJ9Afo/b5gtoiRJ0jSsuFhV1deAryZ5cD90BHA1cAFwQj92AnD+TAklSZImYsOMj/914F1J7gR8ETiRrqydk+Qk4Frg6TO+hiRJ0iTMVKyq6jJg0w5uOmKW55UkSZoid16XJElqxGIlSZLUiMVKkiSpEYuVJElSIxYrSZKkRixWkiRJjVisJEmSGrFYSZIkNWKxkiRJasRiJUmS1IjFSpIkqRGLlSRJUiMWK0mSpEYsVpIkSY1YrCRJkhqxWEmSJDVisZIkSWrEYiVJktSIxUqSJKkRi5UkSVIjFitJkqRGLFaSJEmNWKwkSZIasVhJkiQ1YrGSJElqxGIlSZLUiMVKkiSpEYuVJElSIxYrSZKkRixWkiRJjVisJEmSGrFYSZIkNTJzsUqyR5LPJvmr/vjAJBcn2ZrkvUnuNHtMSZKk8WsxY/Vi4JpFx68F3lhVDwK+BZzU4DUkSZJGb6ZilWR/4Cjgbf1xgCcD5/Z3ORs4dpbXkCRJmopZZ6z+GPht4Mf98X2BG6vqlv54G7DfjK8hSZI0CSsuVkl+Abihqi5Z4eNPTrIlyZb5+fmVxpAkSRqNWWasHg8cneTLwHvoLgG+CdgryYb+PvsD1+3owVW1uao2VdWmubm5GWJIkiSNw4qLVVW9vKr2r6qNwDOB/11VzwY+BBzX3+0E4PyZU0qSJE3Aauxj9TLgJUm20q25OnMVXkOSJGl0Nix9l6VV1YeBD/c/fxE4tMXzajw2nvqBVXvuL59+1Ko9tyRJa8md1yVJkhqxWEmSJDVisZIkSWrEYiVJktSIxUqSJKkRi5UkSVIjFitJkqRGLFaSJEmNWKwkSZIasVhJkiQ1YrGSJElqxGIlSZLUiMVKkiSpEYuVJElSIxYrSZKkRixWkiRJjVisJEmSGrFYSZIkNWKxkiRJasRiJUmS1IjFSpIkqRGLlSRJUiMWK0mSpEYsVpIkSY1YrCRJkhqxWEmSJDWyYegA0mraeOoHVvX5v3z6Uav6/JKkaXHGSpIkqRGLlSRJUiMWK0mSpEYsVpIkSY1YrCRJkhqxWEmSJDWy4mKV5P5JPpTk6iRXJXlxP36fJBcm+UL/573bxZUkSRqvWWasbgF+q6oOAQ4DXpjkEOBU4KKqOgi4qD+WJEla91ZcrKrq+qq6tP/528A1wH7AMcDZ/d3OBo6dNaQkSdIUNFljlWQj8EjgYmCfqrq+v+lrwD47eczJSbYk2TI/P98ihiRJ0qBmLlZJ7g68D/iNqvrHxbdVVQG1o8dV1eaq2lRVm+bm5maNIUmSNLiZilWSO9KVqndV1Xn98NeT7Nvfvi9ww2wRJUmSpmGWTwUGOBO4pqresOimC4AT+p9PAM5feTxJkqTp2DDDYx8PHA9ckeSyfuwVwOnAOUlOAq4Fnj5bREmSpGlYcbGqqo8D2cnNR6z0eSVJkqbKndclSZIasVhJkiQ1YrGSJElqxGIlSZLUiMVKkiSpkVm2W5C0ijae+oFVe+4vn37Uqj23JN2eOWMlSZLUiMVKkiSpEYuVJElSI66xktTUaq4NA9eHSRo3Z6wkSZIasVhJkiQ1YrGSJElqxGIlSZLUiMVKkiSpET8VKEk9d7uXNCtnrCRJkhqxWEmSJDVisZIkSWrEYiVJktSIxUqSJKkRi5UkSVIjFitJkqRGLFaSJEmNuEGoJE3cam5sCm5uKu0OZ6wkSZIaccZKkjQYv0ZI640zVpIkSY1YrCRJkhqxWEmSJDVisZIkSWrEYiVJktSIxUqSJKmRVStWSY5M8vkkW5OculqvI0mSNBarso9Vkj2A/wI8BdgGfCbJBVV19Wq8niRJa2nKu91Pde+wqfydr9aM1aHA1qr6YlX9AHgPcMwqvZYkSdIopKraP2lyHHBkVT2vPz4eeGxVvWjRfU4GTu4PHwx8vnmQzt7AN1bpuVfbVLNPNTdMN/tUc8N0s081N0w3+1Rzw3SzTzU3rG72A6pqbkc3DPaVNlW1Gdi82q+TZEtVbVrt11kNU80+1dww3exTzQ3TzT7V3DDd7FPNDdPNPtXcMFz21boUeB1w/0XH+/djkiRJ69ZqFavPAAclOTDJnYBnAhes0mtJkiSNwqpcCqyqW5K8CPgbYA/g7VV11Wq81jKs+uXGVTTV7FPNDdPNPtXcMN3sU80N080+1dww3exTzQ0DZV+VxeuSJEm3R+68LkmS1IjFSpIkqRGLlSRJUiOD7WMlDSnJo3Z1e1VdulZZNB1JDqyqLy01Njb914ydUlVvHDqLtFqS3Bn4ZWAji/pNVf3emuZYz4vXk9wTqKr69tBZ1rP+pP2/qupJQ2dZriQf2sXNVVVPXrMwM+gL4uFAAX87hUKY5J8CLwUO4LYnv9H/nSe5tKoetd3YJVX16KEyLVeST1fVoUPn2F1JDgOuWjiP9+f1f1ZVFw+bbGlJfraqrhg6x+5KcjndV9G9t6r+z9B5livJB4GbgEuAHy2MV9Xr1zLHupyxSvIY4O3APbrD3Ag8t6ouGTbZ0pJ8m+6X5GI3AVuA36qqL659ql2rqh8l+XGSe1XVTUPnWY4plcCdSfJK4FeA8/qhs5L8ZVX9/oCxluMvgf8K/CmLTn5jluRg4CHAvZL80qKb7gnsOUyq3fa3Sf4z8F7g5oXBCZTxM4DFZfb/7WBsrN7Sz6K8A3jXVM6PwNOAZwDnJPkx3b+Zc6rqK8PGWtL+VXXk0CHW5YxV37ZfWFUf648PB95SVQ8bNtnSkrwa2Ab8BRC6zVUfCFwKvKCqnjhcup1Lcj7wSOBCbnvSPmWwUMuQ5Dk7Gq+qd651lt2V5PPAw6vqe/3xXYDLqurBwybbtanM8CyW5BjgWOBobrvZ8beB91TVJwYJtht2Mks7+tnZJJdV1SO2G7t8CudzgCQHAc+lexP0aeCsqrpw2FTL1+f/j8Czq2qPofPsSpLNwJuHniVcr8Xqs1X1yO3GfmoKf4yS/F1VPXy7scuq6hE7um0skpywo/GqOnuts+yOJG9edLgncARwaVUdN1CkZet/Uf5iVd3YH+8FnDeBX5SvAm4A3g98f2G8qr45VKblSvK4qvrk0DluT5KcB3yYbpYK4N8BT6qqYwcLtZv65RLHAn8C/CPdm+ZXVNV5u3zggJIcQDdr9Qy6meX3rvUltd2V5GrgQcCX6M4toXvzsKYlfF1eCgQ+kuStwLvpLqs9A/jwwoLlkU99fyfJ04Fz++PjgO/1P4+2BY+9QO1MVf364uO+nLxnoDjL0pfBortEfFWSC/vjp9C9Ix67hRL+0kVjBTxggCy76/lJrllUZu8NvL6qnjtwriUluS9wGreuyfs48HtV9Q+DBlva8+kKyX+gy30RcPKgiZYpycOAE4Gj6Gbzn1ZVlya5H/BJbr2MPypJLgbuSHfZ/lfGuARlJ546dABYvzNWk12YnOQBwJuAx9GdRD4F/Cbdl1g/uqo+PmC8neqni/8QOIRFa06qagq/LH8iyR2BK8d8OW1ns4MLplpyp2Ans+E/NTZGfQH/KPDn/dCzgSdW1c8Nl2p9S/IR4G3AuVX13e1uO76q/myYZLuW5MFV9fmhc6xUkp/htr+H1nRt2LosVlp7ST5O9274jXQLH08E7lBVrxw02BKS/HdunQncg64YnlNVLxsu1fL166r+yZROghNf1/Z3dGXkW/3xfYCPVNXPDptsaUmurKqHbjd2xdiz958iPQPYp6oe2s8CHT2BD2lMVpJ70Z3Pn9APfYRudnPUi++THA28Hrgf3XKDA4Brquoha5pjPRarJPsAfwDcr6qemuQQ4HFVdebA0ZaU5Cx2cMlv7JcaFhYkLz5RT2GRcpJ/wa1/37cA11bVdQNGWrYkTwNeB9ypqg5M8gi6k9/RA0fbpYmva3sO8DvAOXTrN44DXjPWmYfFkryB7lLxOf3QccChVfXvh0u1tH7W56XAWxdmBndUEsdoqjP5Sd4HXAkszH4fT/dBmV/a+aOG17/xeTLd9j+PTPIk4Fer6qS1zLFe11i9AziL7gQI8Pd0HxcdfbEC/mrRz3sCvwj834Gy7I7vJ7kD8IUkL6K7dHn3gTPtVJKPV9XhdH/fRfdLEqCSFPBN4I+q6i1DZVyGVwGH0i3spaou6y8lj9oU17UtqKp3JtkKbKL7d3PihBaz/xvgN4CFErgHcHOSf0u3ROKegyXbtbtW1aeTLB67Zagwu+ksbp3JfxL9TP6giZbngVX1y4uOfzfJZYOlWb4fVtU/JLlDkjtU1YeS/PFah1ivxWrvqjonycsBquqWJJPYL6eq3rf4OMm76RaZjt2LgbsCpwCvpjuJ7PCSzxj0pYqquseObu8X+n4CGHOx+mFV3bTdL5wfDxVmBjcDBw4dYjmSvBh4Ht2i4wBvTfKnVfXmXT9yeFV1j/7S5UHcdvbkI8OlWpZvJHkg/cxykuOA64eNtGx3qaqLkqSqrgVeleQSYNRLJIDvJjl8YU1vkscD313iMWNwY5K7Ax8D3pXkBhZt/7NW1muxurn/xbjwH+JhdJ+gmqKDgJ8ZOsQyFN074QPoPk0C3QaQk9hrZnv9u54nDp1jCVcl+dfAHv0lh1PoyuCo7Wxd23CJdstJwGFVdTNAktfSfbpr9MUqyfPo3gDtD1wGHEb37+WIIXMtwwuBzcDBSa6j+yj9s4eNtGyTmslf5AXA2f1aK4BvceunecfsaLpP0b8Y+FW6DXx/d61DrNc1Vo+iO9E9lO468RxwXFVdPmiwZchtd14v4OvAqWPe7wR+slnlS4ErWDRr0r9LU0NJ/qyqjk/yCuBuwL+kmz35G+DVCxuGjtXE17VdATxm0aasewKfGfsCcLg1O/Cpfl+8g4E/GOu6mSQv2W7oLnSX0W4GqKo3rHmo3ZTuW0CuAfaim8m/J/CfauRfx5Nut/jj6Dan3otuYqJqjb9zb7kWlnZs9/tzYSr/x6zx0o71OmP1QLr9LO5P94WMj2Ui/193Ml0/hfY7X1UXLH03NfDofh+cZ9Bdcl28ad9duXXfs1FZJ+vazgIuTvL+/vhYprF2E+B7VfW9JCS5c1V9LslotxWh+0oygAfTFcLz6f7NHM809muD6c7knw/cSPeNH6N/0zO2pR3rdcbq8qp6WLqvsnk13SenXllVjx042pJ2Ml3/yTHvvQWQ5AjgWXSb9y3eTXvUM21TlOQUuqn6B3Dbk97CLsOjX8C+IwsnvzHvIQY/mRE/vD/8WFV9dsg8y9WXwRPpFrA/me7yzh2r6ucHDbaEJB8Fjqpbv4T5HsAHquoJu37k8KY6kz+VT13ujiT7VtWarM1br8Xqs/1HLf8QuKKq/mJCm/hNarp+QZI/Bw4GruLWE0iNfZuIKUtyRlW9YOgcLa3lye/2rL8cey/gg1X1g6Hz7EpfTh5WVd/vj+8MXD72Ag63maWdlIzkO/emahKXx1bgunRfafMU4LX9f4hT+IgrTG+6fsFjpnCiW0/WW6kCsFStjQl8EnCxdwKf3u7y6zuGi7NbTkvyNqY3k3848GtJBv3Ovalar8Xq6cCRwOuq6sYk+3Lb7yUbs239vj7/DbgwybeAUU8b9z6R5JCqunroIJLWj6p6TZK/Bv55P3TiVC6/0l16PZhufdVPZvIZ6XcELjKK79ybqnV5KXC9mNh0/TV0HxrwHY4k0V3GdCb/9me9zlitCxObrj9y6ACSNDLO5N8OOWMlSdIqcCb/9sliJUnSKkhywI7Gx77dgmZjsZIkSWpkKlsQSJIkjZ7FSpIkqRGLlSRJUiMWK0mSpEb+P24iGuxT4fIkAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_data['month'].value_counts().plot(kind='bar', figsize=(10,5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "LSRX1fskMajw",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 349
    },
    "id": "LSRX1fskMajw",
    "outputId": "6298b5ee-eecf-4838-90d6-75bf1d8bcb9b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f8c8d490f90>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlAAAAE7CAYAAAASOb9BAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAUi0lEQVR4nO3dfbDld10f8PeHxAgiBQLbJRJgI2bCpK08XREE25JIixOHZCpDYRS3mnZbWxgotXa1Yx2qM4apBbHTwVkNuEUUwlMTiFIzWyiFMuBuSBEIaWBNSjJ5WJ4KAgWDn/5xfttcMrvZ870POefkvl4zd87v6eS+5ze5e97n9/D9VXcHAID5PWDRAQAAVo0CBQAwSIECABikQAEADFKgAAAGKVAAAINOvy9/2SMf+cjes2fPffkrAQA25MiRI5/r7l0nWnefFqg9e/bk8OHD9+WvBADYkKq6+WTrTnkKr6rOq6rr1v18uapeXlVnVtU1VXXj9PrwrY0NALCcTlmguvuG7n5Sdz8pyVOTfC3JO5PsT3Kou89NcmiaBwC43xu9iPzCJJ/p7puTXJzk4LT8YJJLtjIYAMCyGi1QL0zyB9P07u6+bZq+PcnuE72hqvZV1eGqOnzs2LENxgQAWB5zF6iqOiPJ85K89Z7revZE4hM+lbi7D3T3Wnev7dp1wgvZAQBWysgRqB9Ncm133zHN31FVZyXJ9HrnVocDAFhGIwXqRbn79F2SXJVk7zS9N8mVWxUKAGCZzVWgqurBSZ6T5B3rFl+W5DlVdWOSH5nmAQDu9+YaSLO7v5rkEfdY9vnM7soDANhRPAsPAGCQAgUAMOg+fRbeVtuz/+pFR7hXN1120aIjAADbwBEoAIBBChQAwCAFCgBgkAIFADBIgQIAGKRAAQAMUqAAAAYpUAAAgxQoAIBBChQAwCAFCgBgkAIFADBIgQIAGKRAAQAMUqAAAAYpUAAAgxQoAIBBChQAwCAFCgBgkAIFADBIgQIAGKRAAQAMUqAAAAYpUAAAgxQoAIBBChQAwKC5ClRVPayq3lZVn6qq66vqGVV1ZlVdU1U3Tq8P3+6wAADLYN4jUK9N8p7ufkKSJya5Psn+JIe6+9wkh6Z5AID7vVMWqKp6aJK/meTyJOnub3b3l5JcnOTgtNnBJJdsV0gAgGVy+hzbnJPkWJI3VNUTkxxJ8rIku7v7tmmb25PsPtGbq2pfkn1J8tjHPnbTgdkae/ZfvegI9+qmyy5adAQAOKl5TuGdnuQpSV7X3U9O8tXc43Rdd3eSPtGbu/tAd69199quXbs2mxcAYOHmKVC3JLmluz88zb8ts0J1R1WdlSTT653bExEAYLmcskB19+1JPltV502LLkzyySRXJdk7Ldub5MptSQgAsGTmuQYqSV6a5E1VdUaSo0l+OrPydUVVXZrk5iQv2J6IAADLZa4C1d3XJVk7waoLtzYOAMDyMxI5AMAgBQoAYNC810AB6xhHC2BncwQKAGCQAgUAMEiBAgAYpEABAAxSoAAABilQAACDFCgAgEEKFADAIAUKAGCQAgUAMEiBAgAYpEABAAxSoAAABilQAACDFCgAgEEKFADAIAUKAGCQAgUAMOj0RQcAdp49+69edIR7ddNlFy06ArDkHIECABikQAEADFKgAAAGKVAAAIMUKACAQQoUAMAgBQoAYNBc40BV1U1JvpLkW0nu6u61qjozyVuS7ElyU5IXdPcXtycmAMDyGDkC9ezuflJ3r03z+5Mc6u5zkxya5gEA7vc2cwrv4iQHp+mDSS7ZfBwAgOU3b4HqJH9cVUeqat+0bHd33zZN355k94neWFX7qupwVR0+duzYJuMCACzevM/Ce1Z331pVfzXJNVX1qfUru7urqk/0xu4+kORAkqytrZ1wGwCAVTLXEajuvnV6vTPJO5M8LckdVXVWkkyvd25XSACAZXLKAlVVD66qhxyfTvJ3knw8yVVJ9k6b7U1y5XaFBABYJvOcwtud5J1VdXz73+/u91TVnyS5oqouTXJzkhdsX0wAgOVxygLV3UeTPPEEyz+f5MLtCAUAsMyMRA4AMEiBAgAYpEABAAxSoAAABilQAACDFCgAgEEKFADAIAUKAGCQAgUAMEiBAgAYpEABAAxSoAAABilQAACDFCgAgEEKFADAIAUKAGCQAgUAMEiBAgAYpEABAAxSoAAABilQAACDTl90AADG7Nl/9aIjnNRNl1206Ahwn3AECgBgkAIFADBIgQIAGKRAAQAMUqAAAAYpUAAAgxQoAIBBc48DVVWnJTmc5Nbu/rGqOifJm5M8IsmRJC/u7m9uT0wA2LxlHkMrMY7WKhk5AvWyJNevm39Vktd09/cl+WKSS7cyGADAspqrQFXV2UkuSvI703wluSDJ26ZNDia5ZDsCAgAsm3mPQP1Gkp9P8pfT/COSfKm775rmb0ny6C3OBgCwlE5ZoKrqx5Lc2d1HNvILqmpfVR2uqsPHjh3byH8CAGCpzHME6plJnldVN2V20fgFSV6b5GFVdfwi9LOT3HqiN3f3ge5e6+61Xbt2bUFkAIDFOmWB6u5f6O6zu3tPkhcm+a/d/RNJ3pvk+dNme5NcuW0pAQCWyGbGgfpXSV5RVZ/O7Jqoy7cmEgDAcpt7HKgk6e73JXnfNH00ydO2PhIAwHIzEjkAwCAFCgBg0NApPABg5/IonLs5AgUAMEiBAgAYpEABAAxSoAAABilQAACDFCgAgEEKFADAIAUKAGCQAgUAMEiBAgAYpEABAAxSoAAABilQAACDFCgAgEEKFADAIAUKAGCQAgUAMEiBAgAYpEABAAxSoAAABilQAACDFCgAgEEKFADAIAUKAGCQAgUAMEiBAgAYdMoCVVUPrKqPVNX/rKpPVNUrp+XnVNWHq+rTVfWWqjpj++MCACzePEegvpHkgu5+YpInJXluVT09yauSvKa7vy/JF5Ncun0xAQCWxykLVM/8+TT7HdNPJ7kgydum5QeTXLItCQEAlsxc10BV1WlVdV2SO5Nck+QzSb7U3XdNm9yS5NHbExEAYLnMVaC6+1vd/aQkZyd5WpInzPsLqmpfVR2uqsPHjh3bYEwAgOUxdBded38pyXuTPCPJw6rq9GnV2UluPcl7DnT3Wnev7dq1a1NhAQCWwTx34e2qqodN0w9K8pwk12dWpJ4/bbY3yZXbFRIAYJmcfupNclaSg1V1WmaF64rufndVfTLJm6vqV5N8NMnl25gTAGBpnLJAdffHkjz5BMuPZnY9FADAjmIkcgCAQQoUAMAgBQoAYJACBQAwSIECABikQAEADFKgAAAGKVAAAIMUKACAQQoUAMAgBQoAYJACBQAwSIECABikQAEADFKgAAAGKVAAAIMUKACAQQoUAMAgBQoAYJACBQAwSIECABikQAEADFKgAAAGKVAAAIMUKACAQQoUAMAgBQoAYJACBQAwSIECABh0ygJVVY+pqvdW1Ser6hNV9bJp+ZlVdU1V3Ti9Pnz74wIALN48R6DuSvIvuvv8JE9P8s+q6vwk+5Mc6u5zkxya5gEA7vdOWaC6+7buvnaa/kqS65M8OsnFSQ5Omx1Mcsl2hQQAWCZD10BV1Z4kT07y4SS7u/u2adXtSXZvaTIAgCU1d4Gqqu9O8vYkL+/uL69f192dpE/yvn1VdbiqDh87dmxTYQEAlsFcBaqqviOz8vSm7n7HtPiOqjprWn9WkjtP9N7uPtDda929tmvXrq3IDACwUPPchVdJLk9yfXe/et2qq5Lsnab3Jrly6+MBACyf0+fY5plJXpzkT6vqumnZLya5LMkVVXVpkpuTvGB7IgIALJdTFqju/kCSOsnqC7c2DgDA8jMSOQDAIAUKAGCQAgUAMEiBAgAYpEABAAxSoAAABilQAACDFCgAgEEKFADAIAUKAGCQAgUAMEiBAgAYpEABAAxSoAAABilQAACDFCgAgEEKFADAIAUKAGCQAgUAMEiBAgAYpEABAAxSoAAABilQAACDFCgAgEEKFADAIAUKAGCQAgUAMEiBAgAYpEABAAw6ZYGqqtdX1Z1V9fF1y86sqmuq6sbp9eHbGxMAYHnMcwTqd5M89x7L9ic51N3nJjk0zQMA7AinLFDd/f4kX7jH4ouTHJymDya5ZItzAQAsrY1eA7W7u2+bpm9PsnuL8gAALL1NX0Te3Z2kT7a+qvZV1eGqOnzs2LHN/joAgIXbaIG6o6rOSpLp9c6TbdjdB7p7rbvXdu3atcFfBwCwPDZaoK5Ksnea3pvkyq2JAwCw/OYZxuAPknwoyXlVdUtVXZrksiTPqaobk/zINA8AsCOcfqoNuvtFJ1l14RZnAQBYCUYiBwAYpEABAAxSoAAABilQAACDFCgAgEEKFADAIAUKAGCQAgUAMEiBAgAYpEABAAxSoAAABilQAACDFCgAgEEKFADAIAUKAGCQAgUAMEiBAgAYpEABAAxSoAAABilQAACDFCgAgEEKFADAIAUKAGCQAgUAMEiBAgAYpEABAAxSoAAABilQAACDFCgAgEGbKlBV9dyquqGqPl1V+7cqFADAMttwgaqq05L8xyQ/muT8JC+qqvO3KhgAwLLazBGopyX5dHcf7e5vJnlzkou3JhYAwPLaTIF6dJLPrpu/ZVoGAHC/Vt29sTdWPT/Jc7v7H07zL07yg939kntsty/Jvmn2vCQ3bDzutntkks8tOsSKsu82x/7bHPtvc+y/jbPvNmfZ99/junvXiVacvon/6K1JHrNu/uxp2bfp7gNJDmzi99xnqupwd68tOscqsu82x/7bHPtvc+y/jbPvNmeV999mTuH9SZJzq+qcqjojyQuTXLU1sQAAlteGj0B1911V9ZIk/yXJaUle392f2LJkAABLajOn8NLdf5jkD7coyzJYiVONS8q+2xz7b3Psv82x/zbOvtucld1/G76IHABgp/IoFwCAQQoUAMAgBQoWpKq+c55lACyfHV+gquq0qvqeqnrs8Z9FZ2LH+NCcyzgJf78bU1W7q+ryqvqjaf78qrp00blglWzqLrxVV1UvTfLLSe5I8pfT4k7y/QsLtQKq6je6++VV9a7M9te36e7nLSDWyqiqR2X22KMHVdWTk9S06q8k+a6FBVsx/n435XeTvCHJv57m/1eStyS5fFGBVkVVvTcn/nfvggXEWRkn+7w4bhU/N3Z0gUrysiTndffnFx1kxbxxev31haZYXX83yT/IbPT+V69b/pUkv7iIQCvK3+/GPbK7r6iqX0j+/7h+31p0qBXxc+umH5jkx5PctaAsq+T458XfS/KoJL83zb8osy9BK2enF6jPJvk/iw6xarr7SFWdlmRfd//EovOsmu4+mORgVf14d7990XlWmL/fjftqVT0i0xGBqnp67Mu5dPeReyz6YFV9ZCFhVkh3/7ckqap/f49Ht7yrqg4vKNam7PQCdTTJ+6rq6iTfOL6wu1998reQJN39rap6XFWd0d3fXHSeVdTdb6+qi5L8tcy+yR5f/m8Xl2ql+PvduFdk9uitx1fVB5PsSvL8xUZaDVV15rrZByR5apKHLijOKnpwVX1vdx9Nkqo6J8mDF5xpQ3Z6gfrf088Z0w9jjmb27euqJF89vtAH2Hyq6rcyu+bp2Ul+J7MPMN9k5+fvd4O6+9qq+ltJzsvsGrwbuvsvFhxrVRzJ7MhdZXbq7s+SuAB/fv88sy8+RzPbh49L8o8XG2ljjETOsKp6Y3e/uKq+lOQ191zf3a9cQKyVU1Uf6+7vX/f63Un+qLt/eNHZVsm039Ldf77oLKuiqn7qRMu7+z/d11nYeabhWp4wzX6qu79xb9svqx19BMrdFBv21Kr6nsy+/f+HRYdZYf93ev3atD+/kOSsBeZZKVX11zO7oeHMaf5zSX7KQ83n8gPrph+Y5MIk1yZRoOZQVT+UZE/WfYYqn/Opqu/K7BTy47r7H1XVuVV1Xne/e9HZRu3oAhV3U2zUbyU5lOScJOsv/qvMCun3LiLUCnpXVT0syb/L7MOrk/z2YiOtlANJXtHd702Sqvrbme2/H1pkqFXQ3S9dPz/9f/jmBcVZKVX1xiSPT3JdkuN3LnaUz3m9IbPToM+Y5m9N8tYkCtQqcTfFxnT3byb5zap6XXf/7KLzrLBPJfnWdDH5+UmekuQ/LzjTKnnw8fKUJN39vqpayYtRl8BX44vPvNaSnN+uf9mox3f336+qFyVJd3+tqupUb1pGO7pAneBuirW4m2JuytOm/VJ3v7WqnpXkgszGSXldkh9cbKyVcbSqfil3j0v2k5nd2MApTDd+HPeAJOcnuWJBcVbNxzMbx+i2RQdZUd+sqgfl7iE0Hp91d9Gukh1doPLtd1P8RZKb4m4K7jvHD/9flOS3u/vqqvrVRQZaMT+T5JVJjo+l9d+T/PTi4qyURyX5l9P0XZldz/iSxcVZfutG0n5Ikk9OZyvWD5+xciNpL8gvJ3lPksdU1ZuSPDOzgYVXzo6+C6+qXpDkPd395emb7FOS/Ep3X7vgaOwAVfXuzM7/Pyez//e+nuQj3f3EhQZbEVW1ltmjSPbk7i+D3d0e5XIKVXVtdz/lHss+Zt+d3DTsQyV5VZKfX78qyau625HjOVTV7yX5WGb/3h1N8uHu/txiU23MTi9Qx28ff1aSX8nsFMq/8YfAfWG6G+W5Sf60u2+sqrOS/I3u/uMFR1sJVXVDZjeCfDx3Pwsv3X3zwkItuar62ST/NLPrnT6zbtVDknywu39yIcFWiPK5OVX17CQ/PP08PslHk7y/u1+70GAbsNML1Ee7+8lV9WuZfYj9/vFli84G3Luq+kB3P2vROVZJVT00ycOT/FqS/etWfaW7v7CYVKtB+dw606PAfiCzQYT/SZKvd/cT7v1dy2enFyinUGBFVdWFmT2I9FC+/VqUdywsFPdbyufWqKpDmT265UOZXbf4ge6+c7GpNmanFyinUGBFTddSPCHJJ3L3Kbzu7p9ZXCrg3lTVazJ7fuA3knwwyfuTfKi7v77QYBuwowsUsLqq6obuPm/ROYBxVfWQzO6++7kkj+ru71xsonE7fRgDYHX9j6o6v7s/ueggwHyq6iWZXUD+1MyGDnp9ZqfyVo4CBayqpye5rqr+LLPTARXDGMCye2CSVyc50t0r/eg0p/CAlVRVjzvRcsMYAPcFBQoAYNADFh0AAGDVKFAAAIMUKACAQQoUAMAgBQoAYND/A6BZ8GnAYp/1AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_data['day'].value_counts().plot(kind='bar', figsize=(10,5))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "uPDQxTMzO7xu",
   "metadata": {
    "id": "uPDQxTMzO7xu"
   },
   "source": [
    "We can also check the distribution of the area. Area is very skewed, you can see that most values are very close to zero. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "VIIQnLdQPCUX",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 296
    },
    "id": "VIIQnLdQPCUX",
    "outputId": "a42b52cb-f7ff-4c55-b841-853369f1aeb9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f8c8b079390>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEGCAYAAABy53LJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAewklEQVR4nO3de5RdZZ3m8e+TqlzIrUJCCAlBEkyUjigqMeDQzjDtgNitBrthDEqDM47YIyxaW8cFdkurS9dqp73hknZJiy0yo6C0aJqmxeaia3khUBE0XMxQJEAKQ0hCksrFXCr5zR/v3tZJUUmdc2rvOrWrns9ae+1z9u28Z6+TevK+e+/3VURgZmZWr3GtLoCZmVWLg8PMzBri4DAzs4Y4OMzMrCEODjMza0h7qwswHI477rhYsGBBq4thZlYpq1ev3hIRs/svHxPBsWDBAjo7O1tdDDOzSpH09EDL3VRlZmYNcXCYmVlDHBxmZtYQB4eZmTXEwWFmZg1xcJiZWUMcHGZm1hAHR9ncbb2ZjTIOjjIdPAiLFsGXvtTqkpiZFcbBUabubli3Dv72b2H79laXxsysEA6OMq1bl+bbt8NnP9vaspiZFcTBUaY8OM46C667Dg4caG15zMwK4OAo07p10N4Ol1wCu3bB88+3ukRmZkPm4CjTunVw8skwf356v2lTa8tjZlYAB0eZ1q2DU06BE05I7597rrXlMTMrgIOjTA4OMxuFHBxl6emBLVtg4UKYMyctc3CY2Sjg4CjL+vVpfsopMGkSzJjh4DCzUcHBUZb8VtxTTknzE05wcJjZqODgKEv/4Jgzx8FhZqOCg6Ms3d0wZQoce2x67xqHmY0SDo6ybNsGM2f2vXdwmNko4eAoy/bt6YJ47oQTYOdO2L27dWUyMyuAg6Ms27b1NVNB37McfnrczCrOwVGWgWoc4OYqM6s8B0dZHBxmNko5OMpypKYqB4eZVZyDowy9velCeG2NY/ZsGDcONm5sXbnMzArg4ChDT0+a19Y42trS+xdeaE2ZzMwK4uAow7ZtaV5b4wDo6IAdO4a/PGZmBXJwlGH79jR3cJjZKOTgKENe46htqoIUHHmomJlVlIOjDK5xmNkoVmpwSDpf0lpJXZKuHmD9REm3ZutXSVqQLT9X0mpJa7L5H9Xsc0a2vEvSlySpzO/QlDw4BqpxODjMrOJKCw5JbcD1wJuBJcDFkpb02+w9wLaIWAR8AfhMtnwL8NaIeCVwGXBzzT5fAd4LLM6m88v6Dk3zxXEzG8XKrHEsA7oiYl1E7AduAZb322Y5cFP2+jbgjZIUEQ9FxG+z5Y8Cx2S1k7nA9Ii4PyIC+CZwQYnfoTnbt6fbb6dOPXx5R0e6VffQodaUy8ysAGUGx4nAhpr33dmyAbeJiF5gBzCr3zZ/BvwyIvZl23cPckwAJF0uqVNS5+bNm5v+Ek3Juxvp34rW0QERsGvX8JbHzKxAI/riuKRXkJqv3tfovhFxQ0QsjYils2fPLr5wR7Nt24ubqSAFB7i5yswqrczgeBY4qeb9/GzZgNtIagc6gK3Z+/nA7cClEfFkzfbzBzlm6/Xv4DDn4DCzUaDM4HgQWCxpoaQJwApgZb9tVpIufgNcCNwbESFpBvCvwNUR8bN844jYCPRIOiu7m+pS4Aclfofm9O/gMOfgMLNRoLTgyK5ZXAncBTwOfCciHpX0SUlvyza7EZglqQv4KyC/ZfdKYBFwraSHs+n4bN37ga8BXcCTwL+V9R2a5hqHmY1i7WUePCLuBO7st+zamtd7gYsG2O9TwKeOcMxO4LRiS1qw7dtd4zCzUWtEXxyvLF8cN7NRzMFRtL17Yd8+B4eZjVoOjqLloTB9+ovXTZ6cHgx0cJhZhTk4irZ7d5pPm/bidZK7HTGzynNwFC1/Krx/dyO5GTMcHGZWaQ6Oog0WHK5xmFnFOTiK5uAws1HOwVE0B4eZjXIOjqI5OMxslHNwFM3BYWajnIOjaHlwTJky8Pp8MKeI4SuTmVmBHBxFy5/jmDx54PUdHWkEQA/mZGYV5eAo2q5dqbYx7ginNn+ivKdn+MpkZlYgB0fRdu068vUN6HuifOfO4SmPmVnBHBxFc3CY2Sjn4CjaYMGRr/M1DjOrKAdH0VzjMLNRzsFRNAeHmY1yDo6i5XdVHYmDw8wqzsFRtN27XeMws1HNwVG0wZqqpkxJAzo5OMysohwcRRssOKS03sFhZhXl4CjSgQOwb9/RgwNSc5VvxzWzinJwFCnvp2qw4HCNw8wqzMFRpMG6VM9Nm+bgMLPKcnAUycFhZmOAg6NIDg4zGwMcHEXKr3Ec7QFAcHCYWaU5OIrkGoeZjQEOjiI1Ehy+HdfMKsrBUaR6g2PqVNizBw4eLL9MZmYFc3AUqZEaR+32ZmYV4uAoUqPB4escZlZBDo4i7doF48fDhAlH387BYWYV5uAo0mAdHOYcHGZWYQ6OIg02iFPOwWFmFVZqcEg6X9JaSV2Srh5g/URJt2brV0lakC2fJek+SbskfbnfPj/OjvlwNh1f5ndoyGCDOOV8cdzMKqy9rANLagOuB84FuoEHJa2MiMdqNnsPsC0iFklaAXwGeAewF/gYcFo29feuiOgsq+xNq7epKt/GNQ4zq6AyaxzLgK6IWBcR+4FbgOX9tlkO3JS9vg14oyRFxO6I+CkpQKrD1zjMbAwoMzhOBDbUvO/Olg24TUT0AjuAWXUc+5+yZqqPSdJAG0i6XFKnpM7Nmzc3XvpmODjMbAyo4sXxd0XEK4E3ZNOfD7RRRNwQEUsjYuns2bOHp2T1BsfkyTBunIPDzCqpzOB4Fjip5v38bNmA20hqBzqArUc7aEQ8m813At8iNYmNDPUGh8cdN7MKKzM4HgQWS1ooaQKwAljZb5uVwGXZ6wuBeyMijnRASe2SjstejwfeAjxSeMmbVW9wgHvINbPKKu2uqojolXQlcBfQBnw9Ih6V9EmgMyJWAjcCN0vqAl4ghQsAkp4CpgMTJF0AnAc8DdyVhUYbcDfwj2V9h4ZENB4cvh3XzCqotOAAiIg7gTv7Lbu25vVe4KIj7LvgCIc9o6jyFWrv3hQe9TwACG6qMrPKquLF8ZGp3g4Oc26qMrOKcnAUxcFhZmOEg6MoDg4zGyMcHEVxcJjZGFFXcEj6nqQ/keSgORIHh5mNEfUGwT8A7wSekPR3kl5eYpmqqZng2LsXenvLK5OZWQnqCo6IuDsi3gW8FngKuFvSzyX9t+yZCms0OPLt/CyHmVVM3U1PkmYB7wb+B/AQcB0pSP69lJJVze7daV7vcxzu6NDMKqquBwAl3Q68HLgZeGtEbMxW3Spp5I2L0QrNNFWBg8PMKqfeJ8f/MXsK/PckTYyIfRGxtIRyVU8eHJMn17e9g8PMKqrepqpPDbDsF0UWpPJ27Uqh0dZW3/YODjOrqKPWOCSdQBps6RhJrwHyQZOmA3X+13qMaKSDQ3BwmFllDdZU9SbSBfH5wOdrlu8EPlpSmarJwWFmY8RRgyMibgJukvRnEfHPw1Smamo0OHw7rplV1GBNVZdExP8BFkj6q/7rI+LzA+w2NrnGYWZjxGBNVflDCQ38RRyjdu/uC4N6HHOMxx03s0oarKnqq9n8E8NTnArbtQvmzq1/e8n9VZlZJdXbyeH/ljRd0nhJ90jaLOmSsgtXKY02VYGDw8wqqd7nOM6LiB7gLaS+qhYB/6usQlWSg8PMxoh6gyNv0voT4LsRsaOk8lSXg8PMxoh6uxy5Q9JvgN8B/1PSbGBvecWqmN7e1EV6M8Hh23HNrGLq7Vb9auA/AEsj4gCwG1heZsEqJe8Zt9HgmDrVNQ4zq5x6axwAp5Ke56jd55sFl6eaGu0ZN+emKjOroHq7Vb8ZeCnwMHAwWxw4OJI8OOodiyPn4DCzCqq3xrEUWBIRUWZhKisPjkYeAMy3d3CYWcXUe1fVI8AJZRak0vI//s00Ve3bBwcOFF8mM7OS1FvjOA54TNIDwL58YUS8rZRSVc1QahyQgmfmzGLLZGZWknqD4+NlFqLyhnJxHBwcZlYpdQVHRPxE0snA4oi4W9JkoM6h7saAvKmq0RrH9Olp3tNTbHnMzEpUb19V7wVuA76aLToR+H5ZhaqcZmscDg4zq6B6L45fAZwN9ABExBPA8WUVqnKaDY6OjjTf4R5czKw66g2OfRGxP3+TPQToW3NzO3fCpEnQ3sjzlDg4zKyS6g2On0j6KHCMpHOB7wL/Ul6xKqaZDg7BwWFmlVRvcFwNbAbWAO8D7gT+pqxCVY6Dw8zGkHrvqjok6fvA9yNic8llqp6dOxu/owrS8LFtbb44bmaVctQah5KPS9oCrAXWZqP/XTs8xauIZmscUqp1uMZhZhUyWFPVB0l3U70uImZGxEzgTOBsSR8c7OCSzpe0VlKXpKsHWD9R0q3Z+lWSFmTLZ0m6T9IuSV/ut88ZktZk+3xJkur8ruVptsYBDg4zq5zBguPPgYsjYn2+ICLWAZcAlx5tR0ltwPXAm4ElwMWSlvTb7D3AtohYBHwB+Ey2fC/wMeDDAxz6K8B7gcXZdP4g36F8zdY4wMFhZpUzWHCMj4gt/Rdm1znGD7LvMqArItZlt/LewosHf1oO3JS9vg14oyRFxO6I+Cn9RhmUNBeYHhH3Zz31fhO4YJBylM/BYWZjyGDBsb/JdZCeLt9Q8747WzbgNhHRC+wAZg1yzO5BjgmApMsldUrq3Ly55Ov5Q2mqmj7dF8fNrFIGC47TJfUMMO0EXjkcBWxWRNwQEUsjYuns2bPL/TDXOMxsDDnq7bgRMZSODJ8FTqp5Pz9bNtA23dnT6B3A1kGOOX+QYw6v/fvT5IvjZjZG1PsAYDMeBBZLWihpArACWNlvm5XAZdnrC4F7jzbKYERsBHoknZXdTXUp8IPii96AZvupyuXB4cEVzawiGuxcqX4R0SvpSuAuUhfsX4+IRyV9EuiMiJXAjcDNkrqAF0jhAoCkp4DpwARJFwDnRcRjwPuBbwDHAP+WTa0z1OCYPh0OHoQ9exofs9zMrAVKCw6AiLiT1D1J7bJra17vBS46wr4LjrC8EzituFIOUbOj/+Xybkd6ehwcZlYJZTZVjQ3Njjeec39VZlYxDo6hKuIaBzg4zKwyHBxDVVRTlYPDzCrCwTFUQ22qyoePdXCYWUU4OIaqyIvjZmYV4OAYKl8cN7MxxsExVHmNY/Lk5vafNi2Ny+HgMLOKcHAMVd5P1bgmT+W4cSk8HBxmVhEOjqHaubP5Zqrc9OkODjOrDAfHUA2lZ9zcjBmwfXsx5TEzK5mDY6h27Oi7wN2smTPhhReKKY+ZWckcHENVRHDMmuXgMLPKcHAMVVHBsfVow5CYmY0cDo6hKqqpautWj8lhZpXg4Biqomoc+/enMTnMzEY4B8dQHDqUbsctIjjAzVVmVgkOjqHYuTM1LxXRVAUODjOrBAfHUOTPXhRV4/CdVWZWAQ6Oocif9nZTlZmNIQ6OoSgqOPKmKtc4zKwCHBxDkQfHjBlDO46vcZhZhTg4hqKoGsfEiTBlioPDzCrBwTEURQUHuNsRM6sMB8dQFB0crnGYWQU4OIZixw6YMAEmTRr6sfJuR8zMRjgHx1AU0d1Izk1VZlYRDo6hKDo4XOMwswpwcAxFkcGRD+Z06FAxxzMzK4mDYyi2by+2xnHoEPT0FHM8M7OSODiGouimKnBzlZmNeA6OoSgjOLZsKeZ4ZmYlcXAMRZHBMXdumm/cWMzxzMxK4uBo1sGDxQzilJs3L80dHGY2wjk4mrVzZ5oXFRyzZ8O4cfDb3xZzPDOzkjg4mlVUz7i5tjaYM8c1DjMb8RwczSqyn6rcvHmucZjZiFdqcEg6X9JaSV2Srh5g/URJt2brV0laULPummz5Wklvqln+lKQ1kh6W1Flm+Y8qv/spvxuqCHPnusZhZiNeacEhqQ24HngzsAS4WNKSfpu9B9gWEYuALwCfyfZdAqwAXgGcD/xDdrzcf46IV0fE0rLKP6g8OI47rrhjOjjMrALKrHEsA7oiYl1E7AduAZb322Y5cFP2+jbgjZKULb8lIvZFxHqgKzveyJEHx+zZxR1z3jx4/nno7S3umGZmBSszOE4ENtS8786WDbhNRPQCO4BZg+wbwI8krZZ0+ZE+XNLlkjoldW7evHlIX2RA+THzYV+LMHcuRMCmTcUd08ysYFW8OP6HEfFaUhPYFZL+40AbRcQNEbE0IpbOLrJWkNuyBY49FsaPL+6Y+bMcvkBuZiNYmcHxLHBSzfv52bIBt5HUDnQAW4+2b0Tk8+eB22lVE9bmzcVe3wA/PW5mlVBmcDwILJa0UNIE0sXulf22WQlclr2+ELg3IiJbviK762ohsBh4QNIUSdMAJE0BzgMeKfE7HNmWLcVe3wAHh5lVQntZB46IXklXAncBbcDXI+JRSZ8EOiNiJXAjcLOkLuAFUriQbfcd4DGgF7giIg5KmgPcnq6f0w58KyJ+WNZ3OKotW+Dkk4s95pw5ILmpysxGtNKCAyAi7gTu7Lfs2prXe4GLjrDvp4FP91u2Dji9+JI2YfNmOOOMYo/Z3g7HH+8ah5mNaFW8ON56EeU0VUG6QP5s/0tBZmYjh4OjGTt3wv79xV8cB1iwANavL/64ZmYFcXA0o4yH/3KLFsGTT6Zu283MRiAHRzPK6G4kt3hxqs10dxd/bDOzAjg4mpE/NV5GcCxalOZdXcUf28ysAA6OZpTZVLV4cZo/8UTxxzYzK4CDoxll1jjmzYNJk1zjMLMRy8HRjC1bYMIEmDat+GOPG5eaq1zjMLMRysHRjC1bUm0jPcFevEWLXOMwsxHLwdGMMjo4rLV4cbol99Ch8j7DzKxJDo5mdHfDif2HFinQokWwbx9s2DD4tmZmw8zB0YxnnoGXvKS847/sZWn+m9+U9xlmZk1ycDRq9+5yesat9ZrXpHlnZ3mfYWbWJAdHo/LmozJrHB0dcOqp8MAD5X2GmVmTHByNevrpNC+zxgGwbBmsWpV64jUzG0EcHI165pk0L7PGASk4Nm1yn1VmNuI4OBr19NPQ1pae8C7TsmwodTdXmdkI4+Bo1DPPpFtx20sdPBFe9ar0dLqDw8xGGAdHo55+uvzrGwATJ8KrXw0//3n5n2Vm1gAHR6PKfoaj1rnnwi9+AVu3Ds/nmZnVwcHRiIMH08Xq4ahxAFxwQfrMO+4Yns8zM6uDg6MRGzdCb+/w1TjOOAPmz4fbbx+ezzMzq4ODoxFPPZXmw1XjkFKt40c/gj17huczzcwG4eBoxK9+leannTZ8n/n2t8Pvfgc/+MHwfaaZ2VE4OBqxenUaLrbMnnH7O+ec1Onh5z7np8jNbERwcDTil79M1x3KGsBpIOPGwYc+lELrxz8evs81MzsCB0e99u6FRx+F1752+D/70kvh+OPh0592rcPMWs7BUa9HHkl3VLUiOCZNgo9+FO65B7797eH/fDOzGg6Oev3yl2neiuAAuPJKOOssuOqq1PmhmVmLODjqtXo1zJgBCxa05vPb2uDGG9NAUm99K+za1ZpymNmY5+Co109/OvwXxvtbsgS+851U+3nb22DbttaVxczGLAdHPdasgccegz/901aXJNU2vvGNFGSve517zzWzYefgqMe3vpWaii66qNUlSS65BO67Lz0YeOaZ8O53w+OPt7pUZjZGODgGc+hQupPpvPPSw38jxdlnp7D48Ifh1ltTM9Yb3gBf/GLfKIVmZiVwcAzm7rvTGBzvfGerS/Ji06fD3/99CopPfQp6euCDH0x9ab3iFfC+98HNN0NXl5//MLPCKMbAH5SlS5dGZ2dn4zv29KSR+Nrb4de/hsmTiy9c0Z54IvWm+5OfwM9+Bjt2pOWzZqXhaJctS81by5alZcMlIpVl9+50PtvbYerUNGCVmY1IklZHxNL+y0sd/1TS+cB1QBvwtYj4u37rJwLfBM4AtgLviIinsnXXAO8BDgJXRcRd9RyzUB/4AGzYkC5EVyE0ABYvho98JE2HDqWn3e+/H1atStMPf9hX+3jpS1OInHkmLF0Kp58OU6Y09nkRaaCp3/42dTtfO/Vftnfv4ftKaez2U06BRYtSSL/ylWk+kpoFzewwpdU4JLUB/w84F+gGHgQujojHarZ5P/CqiPgLSSuAt0fEOyQtAb4NLAPmAXcDL8t2O+oxB9JUjePAAVixAk49NXX1MVrs3JmeScmDZNWq9Ace0h/yl70M/uAPYM6cNE2fngaTOnAg1RY2bTp8eu65tK6/jg6YO7dvmjcvzadOTcfr7YUXXoD162HdOli7Fp5/vm//OXNSgORh8pKXpBrSrFkpxNva+iYp3SiwZ0/f1NMD27enadu2vte13dOPH5+ezcmnY49N846OFKD5NHlyGv+9lbdij1VH+vt0tL9bRe5TRhn27En/DvOpp+fwaceONN+9O/2+J0xI09SpMG1a+jc5bdrhr2uXFViLb0WNYxnQFRHrsgLcAiwHav/ILwc+nr2+DfiyJGXLb4mIfcB6SV3Z8ajjmMUYPx5uuy39r300mTYt9bh7zjl9y7q707MhDz2UpieeSLWsrVsP/wfQ1tYXKHPmpOsoteGQB8QJJzRXQ9u0Kd36vGZNahpcswauv/7FNZVm5SGQB8C+fekfaCP/eZJSx5MDzY8ULMP5R2k49injeHa4adP6/pO1f3/6rf7ud/XtO25c33+qxo2DLVvgmGMKLV6ZwXEisKHmfTdw5pG2iYheSTuAWdny+/vtm/dlPtgxAZB0OXB59naXpLVNfIciHAdsadFnF+vgwVQ7yWsoxRmec7R7d5qGIiKdh+E3en5H5Rk95yivjTTj0KE05S0Bh/8nrtFzNOCodaVe42iliLgBuKHV5ZDUOVBVz/r4HA3O52hwPkeDK+oclXk77rPASTXv52fLBtxGUjvQQbpIfqR96zmmmZmVqMzgeBBYLGmhpAnACmBlv21WApdlry8E7o10tX4lsELSREkLgcXAA3Ue08zMSlRaU1V2zeJK4C7SrbNfj4hHJX0S6IyIlcCNwM3Zxe8XSEFAtt13SBe9e4ErIuIgwEDHLOs7FKTlzWUV4HM0OJ+jwfkcDa6QczQmHgA0M7PiuMsRMzNriIPDzMwa4uAokaTzJa2V1CXp6laXpxUknSTpPkmPSXpU0l9my2dK+ndJT2TzY7PlkvSl7Jz9WlKLxuodfpLaJD0k6Y7s/UJJq7JzcWt2QwjZTSO3ZstXSVrQynIPF0kzJN0m6TeSHpf0ev+ODifpg9m/s0ckfVvSpDJ+Rw6OkmRdrlwPvBlYAlycdaUy1vQCH4qIJcBZwBXZebgauCciFgP3ZO8hna/F2XQ58JXhL3LL/CVQO7DKZ4AvRMQiYBup7zay+bZs+Rey7caC64AfRsSpwOmkc+XfUUbSicBVwNKIOI10A9EKyvgdRYSnEibg9cBdNe+vAa5pdblaPQE/IPU1thaYmy2bC6zNXn+V1P9Yvv3vtxvNE+mZpHuAPwLuAER6wre9/++JdFfh67PX7dl2avV3KPn8dADr+39P/44OOxd5Txwzs9/FHcCbyvgducZRnoG6XDnxCNuOCVlV+DXAKmBORGzMVj0HzMlej9Xz9kXgI0DeOdosYHtE9Gbva8/DYV31AHlXPaPZQmAz8E9Zc97XJE3Bv6Pfi4hngc8CzwAbSb+L1ZTwO3Jw2LCQNBX4Z+ADEdFTuy7Sf3nG7H3hkt4CPB8Rq1tdlhGsHXgt8JWIeA2wm75mKcC/o+z6znJSyM4DpgDnl/FZDo7yuHuUjKTxpND4vxHxvWzxJklzs/VzgbxP9bF43s4G3ibpKeAWUnPVdcCMrCseOPw8HKmrntGsG+iOiFXZ+9tIQeLfUZ//AqyPiM0RcQD4Hum3VfjvyMFRHnePQrq7hdRDwOMR8fmaVbXdzVxGuvaRL780uyvmLGBHTVPEqBQR10TE/IhYQPqd3BsR7wLuI3XFAy8+RwN11TNqRcRzwAZJL88WvZHUs4R/R32eAc6SNDn7d5efo+J/R62+oDOaJ+CPSQNPPQn8davL06Jz8Iek5oNfAw9n0x+T2lLvAZ4gDdQ1M9tepLvRngTWkO4Qafn3GMbzdQ5wR/b6FFIfbV3Ad4GJ2fJJ2fuubP0prS73MJ2bVwOd2W/p+8Cx/h296Bx9AvgN8AhwMzCxjN+RuxwxM7OGuKnKzMwa4uAwM7OGODjMzKwhDg4zM2uIg8PMzBri4DAzs4Y4OMxaJOtB2axyHBxmJZH0fUmrs/ERLs+W7ZL0OUm/Al4v6RJJD0h6WNJX8zCR9BVJndm+n2jpFzHrx8FhVp7/HhFnAEuBqyTNInU8tyoiTif1C/QO4OyIeDVwEHhXtu9fR8RS4FXAf5L0quEvvtnA2gffxMyadJWkt2evTyINKnSQ1OEjpL6EzgAeTF0LcQx9nfT916yW0k4aZ2IJqasNs5ZzcJiVQNI5pN5KXx8ReyT9mNQ30N6IOJhvBtwUEdf023ch8GHgdRGxTdI3sn3NRgQ3VZmVo4M0LOceSaeShs3t7x7gQknHw+/HYT8ZmE4ab2KHpDmkYVDNRgzXOMzK8UPgLyQ9Thq29P7+G0TEY5L+BviRpHHAAeCKiLhf0kOkXk43AD8bxnKbDcq945qZWUPcVGVmZg1xcJiZWUMcHGZm1hAHh5mZNcTBYWZmDXFwmJlZQxwcZmbWkP8PoXs95aJBMicAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.kdeplot(data=train_data, x='area', color='red')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Mr_SUbRXSTiB",
   "metadata": {
    "id": "Mr_SUbRXSTiB"
   },
   "source": [
    "<a name='3-3'></a>\n",
    "\n",
    "## 3.3 Preparing the Data for the Model\n",
    "\n",
    "Here we will do two things, one is to normalize numerical features and the second is to encode categorical features. We can set up a pipeline to handle that. \n",
    "\n",
    "For simplicity, we will use Scikit-Learn processing functions. \n",
    "\n",
    "I will first separate features and label. We can use a function that can also be applied to test set.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "4fHdD4ND8klK",
   "metadata": {
    "id": "4fHdD4ND8klK"
   },
   "outputs": [],
   "source": [
    "def get_feats_and_labels(data, label):\n",
    "  \"\"\" Take data and label as inputs, return features and labels separated \"\"\"\n",
    "\n",
    "  data_feats = data.drop(label, axis=1)\n",
    "  data_label = data[label]\n",
    "\n",
    "  return data_feats, data_label"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "w2xPKoSO9bI9",
   "metadata": {
    "id": "w2xPKoSO9bI9"
   },
   "source": [
    "Let's use the function created above to get the features and labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "ruS-krcN81nA",
   "metadata": {
    "id": "ruS-krcN81nA"
   },
   "outputs": [],
   "source": [
    "train_feats, train_label = get_feats_and_labels(train_data, 'area')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "-LC74aAlSaOJ",
   "metadata": {
    "id": "-LC74aAlSaOJ"
   },
   "outputs": [],
   "source": [
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.preprocessing import StandardScaler, OrdinalEncoder\n",
    "\n",
    "scaler = StandardScaler()\n",
    "encoder = OrdinalEncoder()\n",
    "\n",
    "\n",
    "# The column transformer requires lists of features\n",
    "\n",
    "num_feats = ['X', 'Y', 'FFMC', 'DMC', 'DC', 'ISI', 'temp', 'RH',\n",
    "       'wind', 'rain']\n",
    "cat_feats = ['month', 'day']\n",
    "\n",
    "# define the pipeline to scale the numeric features and handle categorical features\n",
    "final_pipe = ColumnTransformer([\n",
    "   ('num',scaler , num_feats),    \n",
    "   ('cat', encoder , cat_feats)                        \n",
    "\n",
    "])\n",
    "\n",
    "training_data_prepared = final_pipe.fit_transform(train_feats)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "tYog2gmfWhzq",
   "metadata": {
    "id": "tYog2gmfWhzq"
   },
   "source": [
    "Now, we can see the shape of the transformed dataset. It is a NumPy array. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "I7nb6z5kWAXf",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "I7nb6z5kWAXf",
    "outputId": "a28a1a98-90ee-4925-d420-77b9c34fa62b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(361, 12)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "training_data_prepared.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "gom2N0zkW6Me",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "gom2N0zkW6Me",
    "outputId": "4d4bb290-9dba-4eaa-f074-42f973c53621"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "numpy.ndarray"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(training_data_prepared)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "loeQqU-v1aoo",
   "metadata": {
    "id": "loeQqU-v1aoo"
   },
   "source": [
    "Also let's tranform the test set. Note that for the test set, we don't `fit_transform()`. \n",
    "\n",
    "I will get the features and labels separated first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "NkI8fXYS1vXp",
   "metadata": {
    "id": "NkI8fXYS1vXp"
   },
   "outputs": [],
   "source": [
    "test_feats, test_label = get_feats_and_labels(test_data, 'area')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Rbq6pgYG2GGt",
   "metadata": {
    "id": "Rbq6pgYG2GGt"
   },
   "source": [
    "And now we transform the test features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "lldqbz9-2M_S",
   "metadata": {
    "id": "lldqbz9-2M_S"
   },
   "outputs": [],
   "source": [
    "test_data_prepared = final_pipe.transform(test_feats)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "J_rggNrUG92O",
   "metadata": {
    "id": "J_rggNrUG92O"
   },
   "source": [
    "Let's convert train and test labels to NumPy array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "CiO8zTLNGR3x",
   "metadata": {
    "id": "CiO8zTLNGR3x"
   },
   "outputs": [],
   "source": [
    "train_label = train_label.to_numpy()\n",
    "test_label = test_label.to_numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "gKG6nbrJXOza",
   "metadata": {
    "id": "gKG6nbrJXOza"
   },
   "source": [
    "<a name='3-4'></a>\n",
    "\n",
    "## 3.4 Creating, Compiling and Training a Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "-ckljbee4qxZ",
   "metadata": {
    "id": "-ckljbee4qxZ"
   },
   "source": [
    "Now that our data is prepared, it's time to create a neural network regressor. \n",
    "\n",
    "Like in the first example, we will use the [Sequential API](https://keras.io/api/models/sequential/). \n",
    "\n",
    "Everytime we are creating a model in TensorFlow, we have to specify the input shape. In this example, the input shape will be:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "ttRc3aELAAnO",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ttRc3aELAAnO",
    "outputId": "95603a37-cebf-430f-a676-56a29e850283"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(12,)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_shape = training_data_prepared.shape[1:]\n",
    "input_shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "G8DZ4p9_XZbi",
   "metadata": {
    "id": "G8DZ4p9_XZbi"
   },
   "outputs": [],
   "source": [
    "model = keras.models.Sequential([\n",
    "                                 \n",
    "            # The first layers must specify the input shape always\n",
    "            keras.layers.Dense(12, activation='relu', input_shape=input_shape),\n",
    "            keras.layers.Dense(24, activation='relu'),\n",
    "\n",
    "            # The last layer usually doesn't have activation function in regression\n",
    "            keras.layers.Dense(1)                \n",
    "\n",
    "])\n",
    "\n",
    "# Now we compile the model\n",
    "model.compile(loss='mean_squared_error', optimizer='adam')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hARZnzj5CuhL",
   "metadata": {
    "id": "hARZnzj5CuhL"
   },
   "source": [
    "Let's see the model summary. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "hkPDJF9TCxEb",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hkPDJF9TCxEb",
    "outputId": "a80ac4d6-2851-41a7-e82b-c89d2ad823c1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_1 (Dense)              (None, 12)                156       \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 24)                312       \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 1)                 25        \n",
      "=================================================================\n",
      "Total params: 493\n",
      "Trainable params: 493\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "AIuSKB93Bk2S",
   "metadata": {
    "id": "AIuSKB93Bk2S"
   },
   "source": [
    "Remember, when the model is created in TensorFlow, it is like an empty graphs. We can even visualize it. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "gyxUlkxGBvUD",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 369
    },
    "id": "gyxUlkxGBvUD",
    "outputId": "f1f30c5e-1065-44e3-b3bf-1bbc7abba9d2"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPEAAAFgCAIAAADl5AgMAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3dfVRT5x0H8OcmkJcbcqOyIGoCNWhlBbG6liJiD53HdszVVQgSFSk4erB263qsNq04jqNSR8GyzcF6UOfZ6hkGsUeRCd0qK2tPscdu+AYFRAaVRgy1GRGS8pI8++OuWYrhJUhyLw+/z1/c+9z75Pck39w83CQ3FMYYAUAQAdcFADDFINOANJBpQBrINCCNn+tCfX39W2+9xVUpAEzOypUrd+7c6Vz81nH65s2bFRUVPi8JgMm7cOFCfX296xq/ezc6efKkr+oB4H4lJyePWAPzaUAayDQgDWQakAYyDUgDmQakgUwD0kCmAWkg04A0kGlAGsg0IA1kGpAGMg1IA5kGpIFMA9Lcb6YzMzPlcjlFUZcuXZqSgqaKw+EoKiqKjY31aK9z584pFIqzZ896qapJuHDhwne/+12BQEBR1Ny5c/fv3++zmz516pRGo6EoiqKo4ODg1NRUn930/bjfTB85cuTw4cNTUsoUun79+uOPP75z506r1erRjjy8MkRMTMxnn3325JNPIoRaWlr27t3rs5tOSkpqb28PCwtTKBTd3d3Hjx/32U3fDwLnHpcvX3711Veff/75hx9+2NN9161b19vb+/TTT3ujMFc2m83T1xDf4G1hEzcFmaYo6v47mULLli07derUli1bxGIx17WM6ujRoyaTiesq3OBtYRM3mUxjjAsKCpYsWSIWixUKxe7du11b7XZ7Tk5OSEiIVCqNiooyGAwIoZKSEplMRtP0mTNnEhISGIZRqVRlZWXOverq6qKjo2maZhhm6dKlFotltK6856OPPgoJCaEo6ne/+924Nf/2t7+VSCRBQUHbt2+fN2+eRCKJjY395JNP2NYXX3xRJBIFBweziy+88IJMJqMo6ssvv0QIvfTSSy+//PKNGzcoilq0aBFCqKamhmGYvLy8idTpy8Im4sMPP3zooYcUCoVEIlm6dOl7772HEMrMzGQn4mFhYQ0NDQihjIwMmqYVCkVlZSUa5cF98803aZqWy+Umk+nll19esGBBS0vLBMv4P+yC7RePJzs7m6KogwcPms1mq9VaXFyMEGpoaGBbd+3aJRaLKyoqzGbznj17BALBxYsX2b0QQufPn+/t7TWZTKtXr5bJZIODgxjjvr4+hmHy8/NtNlt3d3diYmJPT88YXU3QY489tmzZsolvjzG+efMmQujQoUPOkY5WM8Y4KytLJpM1NTV9/fXXjY2Njz76qFwu//zzz9nWLVu2zJ0719lzQUEBQogdF8Y4KSkpLCzM2VpVVSWXy3Nzc0cr7KmnnkIImc1mHxeGMWbn02PcaSdPnty3b99XX311586dmJiYwMBAZ1dCofCLL75wbrl58+bKykr277Fz8vOf//zQoUOJiYmfffbZGDeNMdZqtVqt1nWNx5m2Wq00Ta9du9a5hj1CsJm22Ww0Tet0OufGYrF4x44dzlptNhvbxD4T2traMMbXrl1DCFVVVbne0BhdTdBUZdptzRjjrKws1wf74sWLCKFf/vKX7KKn0Rmb20z7prBxM+3qjTfeQAiZTCaM8fvvv48Q2r9/P9vU29u7ePHi4eFh7ElOxnVvpj2ee7S1tVmt1jVr1rhtbWlpsVqtkZGR7KJUKg0ODm5ubr53S5FIhBAaGhpCCGk0mqCgoNTU1H379nV0dHjalc+41nyvRx55hKZpTirkT2H+/v4IIbvdjhD6/ve//+CDD/7hD3/AGCOETpw4odPphEIh8vKD63Gmu7q6EEJKpdJta39/P0Jo79691Dc6OzvHPaEmlUpra2vj4uLy8vI0Go1Op7PZbJPriltisbinp4frKtzwamF/+ctf4uPjlUqlWCx+5ZVXnOspitq+fXt7e/v58+cRQn/6059+8pOfsE1efXA9zrREIkEIDQwMuG1ls15UVOT6WjDikiJuRUREnD171mg06vV6g8FQWFg46a64MjQ09J///EelUnFdyEjeKOwf//hHUVERQujzzz/fsGFDcHDwJ5980tvbm5+f77pZenq6RCI5cuRIS0sLwzChoaHseq8+uB5nOjIyUiAQ1NXVuW1Vq9USicTT9xSNRmNTUxNCSKlUHjhwYMWKFU1NTZPrikMffPABxjgmJoZd9PPzG20y4GPeKOyf//ynTCZDCF29enVoaGjHjh0ajUYikYw4sTt79uyUlJTTp08XFhY+99xzzvVefXA9zrRSqUxKSqqoqDh69KjFYrly5UppaamzVSKRZGRklJWVlZSUWCwWu93e1dV169atsfs0Go3bt29vbm4eHBxsaGjo7OyMiYmZXFc+5nA4zGbz8PDwlStXXnrppZCQkPT0dLZp0aJFX3311enTp4eGhnp6ejo7O113nDNnjtFo7OjouHv37tDQUHV19cTP5fmysHt7Hhoaun379gcffMBmOiQkBCH0/vvvf/3119evX3eeNHR6/vnnBwYGqqqqXN/J8u6D63rwn+C5vLt372ZmZgYGBgYEBMTFxeXk5CCEVCrV5cuXMcYDAwN6vT4kJMTPz499AjQ2NhYXF9M0jRBavHjxjRs3SktLGYZBCIWGhra2tnZ0dMTGxs6ePVsoFM6fPz87O5v979htV+OWV19fv2rVqnnz5rEDDA4Ojo2NraurG3fHQ4cOsSduaZpev3792DVjjLOysvz9/RcsWODn58cwzDPPPHPjxg1nb3fu3HniiSckEsnChQt/9rOfsWfxFy1axJ5T+9e//hUaGiqVSuPi4rq7u8+dOyeXy52nCFxduHAhIiJCIBCwY8nLy/NZYb///e/DwsJGS867777LdqjX6+fMmTNr1qzk5GT21H5YWJjz1CHGePny5a+99tqIcbl9cPPz86VSKUJIrVa/88474z5keErO5QGnrKysOXPmcF2FG3wr7Ic//GF7e7uXOp+Cc3nAFXvSioc4L8w5b7ly5Qr7muCzm55mmW5ubqZGp9PpvLQv8JRer79+/Xpra2tGRsbrr7/u09t2PWjD3GPiXnvtNfadjgceeODkyZNcl/N/PCksOztbIBCo1Wrnm+Fecu/cg8IunxguLy9PSUnB/PsMMQCjYa8/7XrR9Gk29wBgXJBpQBrINCANZBqQBjINSAOZBqSBTAPSQKYBaSDTgDSQaUAayDQgDWQakAYyDUjjd+8q9oNOAEwLFy5ccH59mPWt47RardZqtb4taaaorKw0Go1cV0GgmJiYlStXuq6h4NPSvkFRlMFg2LhxI9eFkA/m04A0kGlAGsg0IA1kGpAGMg1IA5kGpIFMA9JApgFpINOANJBpQBrINCANZBqQBjINSAOZBqSBTAPSQKYBaSDTgDSQaUAayDQgDWQakAYyDUgDmQakgUwD0kCmAWkg04A0kGlAGsg0IA1kGpAGMg1IA5kGpIFMA9JApgFpINOANPA7Ad6ydevWS5cuORc7OjqUSqVMJmMX/f39z549u2DBAo6qI5mb3ygCU2LJkiXHjx93XdPX1+f8Ozw8HALtJTD38JZNmzZRFOW2yd/fPz093bflzCAw9/Ci733ve5cuXXI4HCPWUxTV3t7+wAMPcFEU+eA47UVpaWkCwch7mKKo6OhoCLT3QKa9KCUl5d6DtEAgSEtL46SeGQIy7UXBwcGrV68WCoUj1iclJXFSzwwBmfaurVu3ui4KBIInnnhi7ty5XNUzE0CmvSs5OXnElHpEysGUg0x7F8MwP/jBD/z8/vc+gFAo/PGPf8xtScSDTHtdamqq3W5HCPn5+a1fv16hUHBdEeEg0163fv16qVSKELLb7Vu2bOG6HPJBpr1OIpEkJiYihGiaTkhI4Loc8vHu8x5dXV0ff/wx11VMMbVajRB69NFHKysrua5liqnV6pUrV3JdxbdhnjEYDFzfJcADWq2W68iMxLvjNAsT9ymUffv27d2713kChAzJyclcl+AGzKd9hLxA8xZk2kcg0D4DmQakgUwD0kCmAWkg04A0kGlAGsg0IA1kGpAGMg1IA5kGpIFMA9JApgFpINOANCRkOjMzUy6XUxTleh1RPnA4HEVFRbGxsRPf5dSpUxqNhnIhEomCgoLi4+MLCgrMZrP3qiUGCZk+cuTI4cOHua5ipOvXrz/++OM7d+60Wq0T3yspKam9vT0sLEyhUGCMHQ6HyWQqLy9fuHChXq+PiIj49NNPvVczGUjINA9dvnz51Vdfff755x9++OH76YeiqFmzZsXHxx87dqy8vPz27dvr1q3r7e2dqjqJREimR7sqLleWLVt26tSpLVu2iMXiqepTq9Wmp6ebTKa33357qvok0nTNNMa4oKBgyZIlYrFYoVDs3r3btdVut+fk5ISEhEil0qioKPY7jiUlJTKZjKbpM2fOJCQkMAyjUqnKysqce9XV1UVHR9M0zTDM0qVLLRbLaF3dp5qaGoZh8vLyPN2RvWp1dXX1tBgmZzj+PuQ92Htz3M2ys7Mpijp48KDZbLZarcXFxQihhoYGtnXXrl1isbiiosJsNu/Zs0cgEFy8eJHdCyF0/vz53t5ek8m0evVqmUw2ODiIMe7r62MYJj8/32azdXd3JyYm9vT0jNHVBD322GPLli0bsbKqqkoul+fm5o62l3M+PQKbP7VazZNharVaHn7Hdlpm2mq10jS9du1a5xr2OMRm2maz0TSt0+mcG4vF4h07duBvHmybzcY2sc+EtrY2jPG1a9cQQlVVVa43NEZXE+Q20+MaLdMYY3aGPXZtPhsmPzM9LecebW1tVqt1zZo1bltbWlqsVmtkZCS7KJVKg4ODm5ub791SJBIhhIaGhhBCGo0mKCgoNTV13759HR0dnnblG/39/RhjhmE8qm3aDfM+TctMd3V1IYSUSqXb1v7+foTQ3r17nad4Ozs7xz2hJpVKa2tr4+Li8vLyNBqNTqez2WyT68p7WltbEULh4eGI6GHep2mZaYlEghAaGBhw28pmvaioyPX1qL6+ftxuIyIizp49azQa9Xq9wWAoLCycdFdeUlNTgxBiL1BG8DDv07TMdGRkpEAgqKurc9uqVqslEomn7ykajcampiaEkFKpPHDgwIoVK5qamibXlZd0d3cXFRWpVKpt27Yhcod5/6ZlppVKZVJSUkVFxdGjRy0Wy5UrV0pLS52tEokkIyOjrKyspKTEYrHY7faurq5bt26N3afRaNy+fXtzc/Pg4GBDQ0NnZ2dMTMzkuhpXdXX1uOfyMMZ9fX0OhwNj3NPTYzAYVq1aJRQKT58+zc6n+T9Mznjpf89Jm+C5vLt372ZmZgYGBgYEBMTFxeXk5CCEVCrV5cuXMcYDAwN6vT4kJMTPz499AjQ2NhYXF9M0jRBavHjxjRs3SktL2XCEhoa2trZ2dHTExsbOnj1bKBTOnz8/Ozt7eHh4tK7GLa++vn7VqlXz5s1j7+Tg4ODY2Ni6ujq29dy5c3K5fP/+/ffuWFlZGRUVRdO0SCRif2CAPdERHR2dm5t7584d1405HyY/z3vw7vcRy8vLU1JS+FYVcIu9Xt7Jkye5LuRbpuXcA4AxQKY91tzcTI1Op9NxXeBMBxcm9Fh4eDhMjfgMjtOANJBpQBrINCANZBqQBjINSAOZBqSBTAPSQKYBaSDTgDSQaUAayDQgDWQakAYyDUgDmQak4elnTcvLy7kuAYyvq6tLpVJxXcVIPM10SkoK1yWACdFqtVyXMBLvvo9IKoqiDAbDxo0buS6EfDCfBqSBTAPSQKYBaSDTgDSQaUAayDQgDWQakAYyDUgDmQakgUwD0kCmAWkg04A0kGlAGsg0IA1kGpAGMg1IA5kGpIFMA9JApgFpINOANJBpQBrINCANZBqQBjINSAOZBqSBTAPSQKYBaSDTgDSQaUAayDQgDWQakAYyDUgDmQak4elvXxCgtLTUbDa7rjlz5sy///1v52J6evrcuXN9Xhf54LcvvCUrK6u0tFQsFrOLGGOKoti/h4eHFQpFd3e3v78/dwUSC+Ye3rJp0yaE0MA3BgcHnX8LBIJNmzZBoL0EjtPe4nA45s2bZzKZ3LZ+9NFHq1at8nFJMwQcp71FIBCkpqaKRKJ7m+bNmxcbG+v7kmYIyLQXbdq0aXBwcMRKf3//tLQ059waTDmYe3iXRqNxPdfBunTp0rJlyzipZyaA47R3paWljfhfUKPRQKC9CjLtXampqUNDQ85Ff3//jIwMDuuZCWDu4XVRUVHXrl1z3s+tra2LFy/mtiSywXHa69LS0oRCIUKIoqjly5dDoL0NMu11mzdvttvtCCGhUPjss89yXQ75INNeN3/+/NjYWIqiHA5HcnIy1+WQDzLtC1u3bsUYP/744/Pnz+e6lhkA84zBYOD6LgEe0Gq1XEdmJJ5+1pS8ZB88eDArKysgIIDrQqZSUVER1yW4wdNMb9y4kesSplhsbKxKpeK6iil28uRJrktwA+bTPkJeoHkLMg1IA5kGpIFMA9JApgFpINOANJBpQBrINCANZBqQBjINSAOZBqSBTAPSQKYBaSDTgDQkZDozM1Mul1MUdenSJa5r+Z/c3NyHHnqIYRixWLxo0aJXXnmlr69vIjueOnVKo9FQLkQiUVBQUHx8fEFBwYiL/wK3SMj0kSNHDh8+zHUV31JbW/vTn/60o6Pjyy+/fOONN379619P8JuISUlJ7e3tYWFhCoUCY+xwOEwmU3l5+cKFC/V6fURExKeffurt4qc7EjLNQwEBAVlZWXPmzJHL5Rs3btywYUNNTc3Nmzc97YeiqFmzZsXHxx87dqy8vPz27dvr1q3r7e31Rs3EICTTfLukYlVVFXtND9Z3vvMdhJDVar2fPrVabXp6uslkevvtt++3PqJN10xjjAsKCpYsWSIWixUKxe7du11b7XZ7Tk5OSEiIVCqNiopiv91YUlIik8lomj5z5kxCQgLDMCqVqqyszLlXXV1ddHQ0TdMMwyxdutRisYzWlae++OILqVS6cOFCdrGmpoZhmLy8PE/7SU9PRwhVV1fzc5h8wfWXfEdi781xN8vOzqYo6uDBg2az2Wq1FhcXI4QaGhrY1l27donF4oqKCrPZvGfPHoFAcPHiRXYvhND58+d7e3tNJtPq1atlMtng4CDGuK+vj2GY/Px8m83W3d2dmJjY09MzRlcT19/fL5fLX3zxReeaqqoquVyem5s72i7O+fQIbP7UajVPhqnVann4vfFpmWmr1UrT9Nq1a51r2OMQm2mbzUbTtE6nc24sFot37NiBv3mwbTYb28Q+E9ra2jDG165dQwhVVVW53tAYXU1cdnb2gw8+aLFYJr7LaJnGGLMzbJ4Mk5+ZnpZzj7a2NqvVumbNGretLS0tVqs1MjKSXZRKpcHBwc3NzfduyV7En73uqEajCQoKSk1N3bdvX0dHh6ddjebdd98tLy9/77335HL5xPcaTX9/P8aYYRiPavPBMHllWma6q6sLIaRUKt229vf3I4T27t3rPMXb2dk57v9nUqm0trY2Li4uLy9Po9HodDqbzTa5rpxOnDjxq1/96oMPPnjggQcmProxtLa2IoTCw8MRn4bJN9My0xKJBCE0MDDgtpXNelFRkevrUX19/bjdRkREnD171mg06vV6g8FQWFg46a4QQocOHTp+/Hhtbe0UXk+spqYGIZSQkIB4M0wempaZjoyMFAgEdXV1blvVarVEIvH0PUWj0djU1IQQUiqVBw4cWLFiRVNT0+S6whjr9fqrV6+ePn16Ci+81N3dXVRUpFKptm3bhngwTN6alplWKpVJSUkVFRVHjx61WCxXrlwpLS11tkokkoyMjLKyspKSEovFYrfbu7q6bt26NXafRqNx+/btzc3Ng4ODDQ0NnZ2dMTExk+uqqanpzTffPHz4sL+/v+u73IWFhewG1dXV457Lwxj39fU5HA6McU9Pj8FgWLVqlVAoPH36NDuf5nyY/OWdfz0nb4Ln8u7evZuZmRkYGBgQEBAXF5eTk4MQUqlUly9fxhgPDAzo9fqQkBA/Pz/2CdDY2FhcXEzTNEJo8eLFN27cKC0tZcMRGhra2tra0dERGxs7e/ZsoVA4f/787Ozs4eHh0boau7arV6+6vasLCgrYDc6dOyeXy/fv33/vvpWVlVFRUTRNi0QigUCAvnkrMTo6Ojc3986dO64bcztMzNfzHrz77Yvy8vKUlBS+VQXcYj/Ewrer5k3LuQcAY4BMe6y5uZkanU6n47rAmY6n1+rls/DwcJga8RkcpwFpINOANJBpQBrINCANZBqQBjINSAOZBqSBTAPSQKYBaSDTgDSQaUAayDQgDWQakAYyDUjD08+a8u36d2A0Wq2W6xJG4t13t7q6uj7++GOuq5h6KSkpL7300sqVK7kuZIqp1Wq+DYp3mSYVRVEGg2Hjxo1cF0I+mE8D0kCmAWkg04A0kGlAGsg0IA1kGpAGMg1IA5kGpIFMA9JApgFpINOANJBpQBrINCANZBqQBjINSAOZBqSBTAPSQKYBaSDTgDSQaUAayDQgDWQakAYyDUgDmQakgUwD0kCmAWkg04A0kGlAGsg0IA1kGpAGMg1IA5kGpOHpb18QoLOz0263u665fft2e3u7c3HevHlSqdTndZEPfifAWxISEmpqakZr9fPz6+7uDgwM9GVJMwTMPbxFp9ON9ktLAoFg7dq1EGgvgUx7S2Jior+//2itW7du9WUxMwpk2lvkcvmPfvQjt7H29/d/+umnfV/SDAGZ9qItW7YMDw+PWOnn57dhw4aAgABOSpoJINNetG7dOplMNmKl3W7fsmULJ/XMEJBpLxKLxVqtViQSua4MCAh48sknuSppJoBMe9fmzZsHBwedi/7+/jqdbkTKwdSC89Pe5XA45s6d++WXXzrX/P3vf4+Pj+euIvLBcdq7BALB5s2bnQdmpVK5evVqbksiHmTa6zZt2sROP0QiUVpamlAo5LoiwsHcw+swxqGhoTdv3kQIXbx48ZFHHuG6IsLBcdrrKIpKS0tDCIWGhkKgfYB3n8urr69/6623uK5iilksFoSQTCZLTk7mupYptnLlyp07d3Jdxbfw7jh98+bNiooKrquYYgzDKBQKlUrFdSFT7MKFC/X19VxXMRLvjtOskydPcl3CFHvvvfeeeuoprquYYvx82eHdcZpU5AWatyDTgDSQaUAayDQgDWQakAYyDUgDmQakgUwD0kCmAWkg04A0kGlAGsg0IA1kGpAGMg1IQ0KmMzMz5XI5RVGXLl3iupb/yc/PDw8Pl0qlMpksPDz8F7/4Bfu1gHGdOnVKo9FQLkQiUVBQUHx8fEFBgdls9nblBCAh00eOHDl8+DDXVXzLhx9++Nxzz33++ee3b99+/fXX8/PztVrtRHZMSkpqb28PCwtTKBQYY4fDYTKZysvLFy5cqNfrIyIiPv30U28XP92RkGkeEolEL7zwglKpDAgISE5OfuaZZ/72t7/dunXL034oipo1a1Z8fPyxY8fKy8tv3769bt263t5eb9RMDEIyPdqVnrny7rvvSiQS5+KCBQsQQn19fffTp1arTU9PN5lMb7/99v3WR7TpmmmMcUFBwZIlS8RisUKh2L17t2ur3W7PyckJCQmRSqVRUVEGgwEhVFJSIpPJaJo+c+ZMQkICwzAqlaqsrMy5V11dXXR0NE3TDMMsXbqUnQG77cpT169fnzVrVmhoKLtYU1PDMExeXp6n/aSnpyOEqqur+TlMvsA8w96b426WnZ1NUdTBgwfNZrPVai0uLkYINTQ0sK27du0Si8UVFRVms3nPnj0CgeDixYvsXgih8+fP9/b2mkym1atXy2SywcFBjHFfXx/DMPn5+Tabrbu7OzExsaenZ4yuJmJwcLCrq+vQoUNisfidd95xrq+qqpLL5bm5uaPt6JxPj8DmT61W82SYWq1Wq9VO8N7wmWmZaavVStP02rVrnWvY4xCbaZvNRtO0TqdzbiwWi3fs2IG/ebBtNhvbxD4T2traMMbXrl1DCFVVVbne0BhdTcTcuXMRQoGBgb/5zW/YSE3QaJnGGLMzbJ4Mk5+ZnpZzj7a2NqvVumbNGretLS0tVqs1MjKSXZRKpcHBwc3NzfduyV7GbmhoCCGk0WiCgoJSU1P37dvX0dHhaVdu3bx502Qy/fnPf/7jH/+4fPlyk8nkwSDd6e/vxxgzDONRbd4eJt9My0x3dXUhhJRKpdvW/v5+hNDevXudp3g7OzutVuvYfUql0tra2ri4uLy8PI1Go9PpbDbb5Lpy8vf3VyqVTz755IkTJxobG9944w0PBulOa2srQig8PBzxaZh8My0zzZ5SGBgYcNvKZr2oqMj19Wgil1aJiIg4e/as0WjU6/UGg6GwsHDSXY2waNEioVDY2Njo6Y4jsD9Ol5CQgHg5TJ6YlpmOjIwUCAR1dXVuW9VqtUQi8fQ9RaPR2NTUhBBSKpUHDhxYsWJFU1PT5Lq6c+fO5s2bXddcv37dbrer1WqP+hmhu7u7qKhIpVJt27YN8WCYvDUtM61UKpOSkioqKo4ePWqxWK5cuVJaWupslUgkGRkZZWVlJSUlFovFbrd3dXWN+36H0Wjcvn17c3Pz4OBgQ0NDZ2dnTEzM5LqSyWR//etfa2trLRbL0NBQQ0PDs88+K5PJnNeVq66uHvdcHsa4r6/P4XBgjHt6egwGw6pVq4RC4enTp9n5NOfD5C8v/e85aRM8l3f37t3MzMzAwMCAgIC4uLicnByEkEqlunz5MsZ4YGBAr9eHhIT4+fmxT4DGxsbi4mKaphFCixcvvnHjRmlpKRuO0NDQ1tbWjo6O2NjY2bNnC4XC+fPnZ2dnDw8Pj9bVuOWtX79+4cKFAQEBYrE4LCxMp9NdvXrV2Xru3Dm5XL5///57d6ysrIyKiqJpWiQSCQQC9M1bidHR0bm5uXfu3HHdmPNh8vO8B++uP11eXp6SksK3qoBb7PXy+HZxw2k59wBgDJBpjzU3N1Oj0+l0XBc40/H0Wr18Fh4eDlMjPoPjNCANZBqQBv0pVzsAAAB1SURBVDINSAOZBqSBTAPSQKYBaSDTgDSQaUAayDQgDWQakAYyDUgDmQakgUwD0kCmAWl4+llT9gsUgOcuXLgQExPDdRUj8e44rVarJ3hZW8C5mJiYlStXcl3FSLz7PiIA94l3x2kA7hNkGpAGMg1IA5kGpPkvg/M1J13adqkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tensorflow.keras.utils import plot_model\n",
    "\n",
    "plot_model(model, to_file='model.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ZHunKbu3CQYt",
   "metadata": {
    "id": "ZHunKbu3CQYt"
   },
   "source": [
    "Before training or fitting the model to the data, a model is nothing other than empty graphs. \n",
    "\n",
    "Now let's train the model. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "KS3MtZzeCoHE",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "KS3MtZzeCoHE",
    "outputId": "2be51460-3ac7-4878-b9ce-48dc2190c191"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "12/12 [==============================] - 1s 17ms/step - loss: 2500.8550 - val_loss: 8183.7715\n",
      "Epoch 2/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 2489.0750 - val_loss: 8166.7290\n",
      "Epoch 3/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2480.3232 - val_loss: 8150.3276\n",
      "Epoch 4/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 2470.4707 - val_loss: 8137.5972\n",
      "Epoch 5/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2462.4663 - val_loss: 8124.1401\n",
      "Epoch 6/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2453.9116 - val_loss: 8110.1162\n",
      "Epoch 7/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 2445.4663 - val_loss: 8095.4053\n",
      "Epoch 8/50\n",
      "12/12 [==============================] - 0s 6ms/step - loss: 2435.2563 - val_loss: 8077.7803\n",
      "Epoch 9/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 2426.2720 - val_loss: 8056.7891\n",
      "Epoch 10/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2415.6418 - val_loss: 8038.3853\n",
      "Epoch 11/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2405.9746 - val_loss: 8021.0791\n",
      "Epoch 12/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2399.9617 - val_loss: 8000.2622\n",
      "Epoch 13/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2388.9194 - val_loss: 7986.3862\n",
      "Epoch 14/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 2383.2515 - val_loss: 7973.9990\n",
      "Epoch 15/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 2377.0486 - val_loss: 7962.8188\n",
      "Epoch 16/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2372.3306 - val_loss: 7954.1953\n",
      "Epoch 17/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2369.2825 - val_loss: 7946.6025\n",
      "Epoch 18/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2364.2703 - val_loss: 7942.8389\n",
      "Epoch 19/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2361.3081 - val_loss: 7942.8628\n",
      "Epoch 20/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2358.3665 - val_loss: 7940.8789\n",
      "Epoch 21/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2354.2932 - val_loss: 7937.1777\n",
      "Epoch 22/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2351.3564 - val_loss: 7934.7852\n",
      "Epoch 23/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2349.0132 - val_loss: 7931.3711\n",
      "Epoch 24/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2345.8608 - val_loss: 7930.4922\n",
      "Epoch 25/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2343.2444 - val_loss: 7931.0679\n",
      "Epoch 26/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 2339.5933 - val_loss: 7927.4663\n",
      "Epoch 27/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2337.6555 - val_loss: 7924.7412\n",
      "Epoch 28/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 2334.4104 - val_loss: 7924.7734\n",
      "Epoch 29/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2332.6509 - val_loss: 7925.0747\n",
      "Epoch 30/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2328.8726 - val_loss: 7924.2202\n",
      "Epoch 31/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2327.3474 - val_loss: 7927.2397\n",
      "Epoch 32/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2323.9705 - val_loss: 7927.1152\n",
      "Epoch 33/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2321.2361 - val_loss: 7925.8413\n",
      "Epoch 34/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2318.6199 - val_loss: 7926.0889\n",
      "Epoch 35/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2317.1487 - val_loss: 7927.2734\n",
      "Epoch 36/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2314.9399 - val_loss: 7932.5273\n",
      "Epoch 37/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2310.1355 - val_loss: 7924.5747\n",
      "Epoch 38/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 2305.5168 - val_loss: 7923.9902\n",
      "Epoch 39/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2304.0037 - val_loss: 7927.2749\n",
      "Epoch 40/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2300.7092 - val_loss: 7922.8140\n",
      "Epoch 41/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2298.9998 - val_loss: 7923.1523\n",
      "Epoch 42/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2296.6914 - val_loss: 7924.2998\n",
      "Epoch 43/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2294.8440 - val_loss: 7925.5903\n",
      "Epoch 44/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2292.0125 - val_loss: 7931.0190\n",
      "Epoch 45/50\n",
      "12/12 [==============================] - 0s 6ms/step - loss: 2290.0403 - val_loss: 7931.6738\n",
      "Epoch 46/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2287.5166 - val_loss: 7933.3862\n",
      "Epoch 47/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2285.0198 - val_loss: 7933.2266\n",
      "Epoch 48/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2282.4080 - val_loss: 7934.9614\n",
      "Epoch 49/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 2281.5061 - val_loss: 7933.1396\n",
      "Epoch 50/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 2276.9304 - val_loss: 7934.7148\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(training_data_prepared, train_label, \n",
    "                    validation_data = (test_data_prepared, test_label), epochs=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fZD8FlqtEPfT",
   "metadata": {
    "id": "fZD8FlqtEPfT"
   },
   "source": [
    "<a name='3-5'></a>\n",
    "\n",
    "## 3.5 Evaluating a Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "TUOlu65YEif6",
   "metadata": {
    "id": "TUOlu65YEif6"
   },
   "source": [
    "After we have trained the model, the next step is to evaluate it. \n",
    "\n",
    "But first off, we can plot the loss versus the epochs to see how it performed. Plotting the model metrics is a fundamental step in performing the error analysis. \n",
    "\n",
    "`loss` and `val_loss` are contained in `history.history` and the number of epochs are in `history.epoch`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "yMg2Zo3rEif-",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 337
    },
    "id": "yMg2Zo3rEif-",
    "outputId": "6fdc75ea-4129-4090-8fb5-dacf1c24f54f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f8c8b06fe50>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlwAAAEvCAYAAACQQh9CAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dfZBldX3n8ff39sP0PDA8OQ7IYMCSiMqsaFrUSiA+bASJCeZB8SE6UiSkDKsmMUQ0SWEMVkyyJRtrs7hUJKKlwizRlU1cDKukiFWK9IyDgESZoMiMCMM8IDD0TPe93/3j/G73uT3dTDfTZ7qZeb+qTp3f+Z2H+zv39Nz53N85557ITCRJktSc1kI3QJIk6VBn4JIkSWqYgUuSJKlhBi5JkqSGGbgkSZIaZuCSJElqWP9CN+DJPOMZz8iTTjppoZshSZK0Xxs2bHg4M1dNN29RB66TTjqJkZGRhW6GJEnSfkXEfTPN85SiJElSwwxckiRJDTNwSZIkNczAJUmS1DADlyRJUsMMXJIkSQ0zcEmSJDXMwCVJktQwA5ckSVLDFvUvzTeu04aRq2Hp0WU4arK85EhomUclSdKBO7wD1+gj8OU/mmFmwNCRvUFs6KiqvM/4yN66JSsNa5IkacLhHbiGjoI/ugee2FmGXZPl0V1T6nfAzvtK/S7I9pNsOGBwBSxZAUuOmCwPHlFNL1lRm7+y1NWHWt3AMog4aG+JJEmaf4d34Gq1YMUzq2EuMmHvY1Xw6gaw+nj0EdjzGOx9tIwfgz2PwuPbe+vae/f/WtHaN4R1y0PduiOnTK+sykNHVvOGVkLfwFN7jyRJ0gE7vAPXUxUxGX448alvZ3xPLZhNHX7aOz3608m63Q/Dzh+Uukdh/In9v1b/0hLIShAbWllOgx4Ny46pXcd2NCw9pve6NsOaJEkHZFaBKyL+APhtIIE7gAuA44FrgWOBDcDbM3NvRCwBPg38HLAdOD8zf1i28wHgQqANvCczvzKve/N007+kGpYfe2DbGd9b9ZiNPlKFsm44mxg/0juvO73zvsnTp9mZeftLjqzauHxVGZ4By54xWV7+jMl5S482oEmSNMV+A1dEnAC8B3hBZj4REeuBNwPnAldk5rUR8QmqIHVlGe/MzOdGxJuBvwLOj4gXlPVeCDwL+H8R8bOZT3oxlGajfxD6j6l6qp6KTgf2PFK7Zq1ct7Z7R1Xevb3qVXt8G+y4F+7/VjU9U0hbsrLWc3bMDOOjewdvNJAkHcJme0qxH1gaEWPAMuAB4NXAW8v8a4APUQWu80oZ4Hrgv0dElPprM3MP8IOI2AycAXzjwHdDB6TVmgw+s9XpVGHs8W2TYezxh0tI29E73vEfsHtnFepmEq3JU5xTh+WrJnvYJnrWjq2W94YCSdLTwH4DV2ZujYj/CvwIeAL4F6pTiLsyc7wstgU4oZRPAO4v645HxCNUpx1PAL5Z23R9HT3dtFolBM3hdGh7vPSe7ZjSmzblDtFur9r2zVVgmymotfpLAHsGLDu23CSwsvfmge61dt3r15asnLxLdHAF9HkZoySpebM5pXg0Ve/UycAu4H8B5zTVoIi4CLgI4NnPfnZTL6OF0NcPK1ZVw1yM760C2ERvWneYMv3Yg5M3GOx9dHbb7h/q/dmOweW9P9sxsLyqG1xWQtry6qc6BleUuuXVMgNLq20NDFU3KPQN2PsmSZowm6/3/xn4QWZuA4iILwA/DxwVEf2ll2sNsLUsv5Xq1r0tEdEPHEl18Xy3vqu+zoTMvAq4CmB4eDifyk7pENM/CCuPr4bZ6nTKz3H8tHaX56NVb9nET3WUcb2859Eq3O36UVU39jjsfRw64/t/zbpoVcGrG8C64/4lVTDb73iw6sHrDtHqnW71Q6uvGmIO175FqwxlvYhaXW2ob3+f1+y2qfb6E+v1TbN9g6ckzSZw/Qh4eUQsozql+BpgBLgZ+E2qOxXXAV8qy99Qpr9R5n8tMzMibgA+FxEfo7po/hTgW/O4L9KkVqv89MXKA99WZvWbaXtL+BrbXYLa7lL3GIyPwtgTZTxa/VTH+J5aXRmPj1b1o7uq8fho1YPXrR8fhfaeA2/zYtINYN2wVi9P1NVCHt2QVg9r9boyDVQ3TndH3XL2ziOqv4epgXWfENuaEiJr5al1zDZE1r4z5tTvj9N9n5xpn0tdRLWdTrv68eWecacad8arun1e70l0g3f3tWYqz7hv0+x3Zhl3auXaODtVudOu/n21x6qhM1amx6txp9QT1ReSgaHypaT+BaU+vWTf9u7znpZy9z3L8r7V3896PUBroPo76ev+vcwwPdOb0nM8ZngvZnrP9nn/ps4v63ffs+77OfFelvewW+55PfZT7pSBWrkzZd4Mf2vTfdmaur/1bXa31Z0/3ZfBqX+b0ep9T3q23+l9jef+Z3jrtdO39SCYzTVct0bE9cBGYBz4NlUP1D8D10bE5aXuk2WVTwKfKRfF76C6M5HMvKvc4fjdsp2LvUNRTwsRkx/kT/VO0LnodKoPxvp/Ap3xKeVa3bT/cU+j+wHdac/wQdcd2lUbuv9x97x2fXp8cp3uf/jZ/Y+/M2V6fHKZqe2fWlf/j2Wi3bX2T8yrhZCqsO8H/D4Bpdv2ThVsO49PeZ+nBJlpw82T/ITKdGLGid729uwzvftaH9cDYKuvFgz7JoNlzKXnc6b/yDu1dnT/k32S9k83rycw1setyXJfN7gMVuWBpdW1ln2DVZjpG6zCDNn7xWR8T3WNZ326O2bK38xM5YnwX3sfJ74U1N5L6P27b4/tO/2U/zub8v70BNzpAvCTvZflPewbqJXL+ze4HPrKz/Z0/za66z1ZeeJLUIueXuupAWgf03wudf9+9+lZr+9ja7J9PaFuSniaKLf3XXdie/TWHXvKUzxG8yNyLt+CDrLh4eEcGRlZ6GZIkvTkOuU//3r42KeHZ8o8T7cfciJiQ2YOTzfPW7QkSTpQrRbgbwlqZv51SJIkNczAJUmS1DADlyRJUsMMXJIkSQ0zcEmSJDXMwCVJktQwA5ckSVLDDFySJEkNM3BJkiQ1zMAlSZLUMAOXJElSwwxckiRJDTNwSZIkNczAJUmS1DADlyRJUsMMXJIkSQ0zcEmSJDXMwCVJktQwA5ckSVLDDFySJEkNM3BJkiQ1zMAlSZLUMAOXJElSwwxckiRJDTNwSZIkNczAJUmS1DADlyRJUsMMXJIkSQ0zcEmSJDXMwCVJktQwA5ckSVLDDFySJEkN22/giojnRcSm2vDTiPj9iDgmIm6KiHvK+OiyfETExyNic0R8JyJeUtvWurL8PRGxrskdkyRJWiz2G7gy83uZeXpmng78HLAb+CJwKfDVzDwF+GqZBngdcEoZLgKuBIiIY4DLgJcBZwCXdUOaJEnSoWyupxRfA/xHZt4HnAdcU+qvAd5QyucBn87KN4GjIuJ44GzgpszckZk7gZuAcw54DyRJkha5uQauNwOfL+XVmflAKf8EWF3KJwD319bZUupmqpckSTqkzTpwRcQg8KvA/5o6LzMTyPloUERcFBEjETGybdu2+dikJEnSgppLD9frgI2Z+WCZfrCcKqSMHyr1W4ETa+utKXUz1ffIzKsyczgzh1etWjWH5kmSJC1Ocwlcb2HydCLADUD3TsN1wJdq9e8odyu+HHiknHr8CvDaiDi6XCz/2lInSZJ0SOufzUIRsRz4JeB3a9UfBdZHxIXAfcCbSv2XgXOBzVR3NF4AkJk7IuIvgNvKch/OzB0HvAeSJEmLXFSXXy1Ow8PDOTIystDNkCRJ2q+I2JCZw9PN85fmJUmSGmbgkiRJapiBS5IkqWEGLkmSpIYZuCRJkhpm4JIkSWqYgUuSJKlhBi5JkqSGGbgkSZIaZuCSJElqmIFLkiSpYQYuSZKkhhm4JEmSGmbgkiRJapiBS5IkqWEGLkmSpIYZuCRJkhpm4JIkSWqYgUuSJKlhBi5JkqSGGbgkSZIaZuCSJElqmIFLkiSpYQYuSZKkhhm4JEmSGmbgkiRJapiBS5IkqWEGLkmSpIYZuCRJkhpm4JIkSWqYgUuSJKlhBi5JkqSGGbgkSZIaNqvAFRFHRcT1EfHvEXF3RLwiIo6JiJsi4p4yProsGxHx8YjYHBHfiYiX1Lazrix/T0Ssa2qnJEmSFpPZ9nD9LXBjZp4KvAi4G7gU+GpmngJ8tUwDvA44pQwXAVcCRMQxwGXAy4AzgMu6IU2SJOlQtt/AFRFHAmcBnwTIzL2ZuQs4D7imLHYN8IZSPg/4dFa+CRwVEccDZwM3ZeaOzNwJ3AScM697I0mStAjNpofrZGAb8A8R8e2I+PuIWA6szswHyjI/AVaX8gnA/bX1t5S6meolSZIOabMJXP3AS4ArM/PFwONMnj4EIDMTyPloUERcFBEjETGybdu2+dikJEnSgppN4NoCbMnMW8v09VQB7MFyqpAyfqjM3wqcWFt/Tambqb5HZl6VmcOZObxq1aq57IskSdKitN/AlZk/Ae6PiOeVqtcA3wVuALp3Gq4DvlTKNwDvKHcrvhx4pJx6/Arw2og4ulws/9pSJ0mSdEjrn+Vy7wY+GxGDwL3ABVRhbX1EXAjcB7ypLPtl4FxgM7C7LEtm7oiIvwBuK8t9ODN3zMteSJIkLWJRXX61OA0PD+fIyMhCN0OSJGm/ImJDZg5PN89fmpckSWqYgUuSJKlhBi5JkqSGGbgkSZIaZuCSJElqmIFLkiSpYQYuSZKkhhm4JEmSGmbgkiRJapiBS5IkqWGzfZaiJEk6xI2NjbFlyxZGR0cXuimL2tDQEGvWrGFgYGDW6xi4JEkSAFu2bOGII47gpJNOIiIWujmLUmayfft2tmzZwsknnzzr9TylKEmSABgdHeXYY481bD2JiODYY4+dcy+ggUuSJE0wbO3fU3mPDFySJGnRWLFixUI3oREGLkmSpIYZuCRJ0qKTmVxyySWcdtpprF27luuuuw6ABx54gLPOOovTTz+d0047jX/7t3+j3W7zzne+c2LZK664YoFbvy/vUpQkSfv48/9zF9/98U/ndZsveNZKLvuVF85q2S984Qts2rSJ22+/nYcffpiXvvSlnHXWWXzuc5/j7LPP5k/+5E9ot9vs3r2bTZs2sXXrVu68804Adu3aNa/tng/2cEmSpEXn61//Om95y1vo6+tj9erV/OIv/iK33XYbL33pS/mHf/gHPvShD3HHHXdwxBFH8JznPId7772Xd7/73dx4442sXLlyoZu/D3u4JEnSPmbbE3WwnXXWWdxyyy388z//M+985zv5wz/8Q97xjndw++2385WvfIVPfOITrF+/nquvvnqhm9rDHi5JkrTonHnmmVx33XW02222bdvGLbfcwhlnnMF9993H6tWr+Z3f+R1++7d/m40bN/Lwww/T6XT4jd/4DS6//HI2bty40M3fhz1ckiRp0fm1X/s1vvGNb/CiF72IiOCv//qvOe6447jmmmv4m7/5GwYGBlixYgWf/vSn2bp1KxdccAGdTgeAv/zLv1zg1u8rMnOh2zCj4eHhHBkZWehmSJJ0WLj77rt5/vOfv9DNeFqY7r2KiA2ZOTzd8p5SlCRJapiBS5IkqWEGLkmSpIYZuCRJkhpm4JIkSWqYgUuSJKlhBi5JkqSGGbgkSdLT0ooVK2ac98Mf/pDTTjvtILbmyRm4JEmSGuajfSRJ0r7+76Xwkzvmd5vHrYXXfXTG2ZdeeiknnngiF198MQAf+tCH6O/v5+abb2bnzp2MjY1x+eWXc955583pZUdHR3nXu97FyMgI/f39fOxjH+NVr3oVd911FxdccAF79+6l0+nwj//4jzzrWc/iTW96E1u2bKHdbvNnf/ZnnH/++Qe02zDLHq6I+GFE3BERmyJipNQdExE3RcQ9ZXx0qY+I+HhEbI6I70TES2rbWVeWvyci1h1w6yVJ0iHj/PPPZ/369RPT69evZ926dXzxi19k48aN3Hzzzbzvfe9jro8l/Lu/+zsigjvuuIPPf/7zrFu3jtHRUT7xiU/w3ve+l02bNjEyMsKaNWu48cYbedaznsXtt9/OnXfeyTnnnDMv+zaXHq5XZebDtelLga9m5kcj4tIy/X7gdcApZXgZcCXwsog4BrgMGAYS2BARN2TmznnYD0mSNJ+epCeqKS9+8Yt56KGH+PGPf8y2bds4+uijOe644/iDP/gDbrnlFlqtFlu3buXBBx/kuOOOm/V2v/71r/Pud78bgFNPPZWf+Zmf4fvf/z6veMUr+MhHPsKWLVv49V//dU455RTWrl3L+973Pt7//vfz+te/njPPPHNe9u1AruE6D7imlK8B3lCr/3RWvgkcFRHHA2cDN2XmjhKybgLmJzZKkqRDwhvf+Eauv/56rrvuOs4//3w++9nPsm3bNjZs2MCmTZtYvXo1o6Oj8/Jab33rW7nhhhtYunQp5557Ll/72tf42Z/9WTZu3MjatWv50z/9Uz784Q/Py2vNNnAl8C8RsSEiLip1qzPzgVL+CbC6lE8A7q+tu6XUzVQvSZIEVKcVr732Wq6//nre+MY38sgjj/DMZz6TgYEBbr75Zu677745b/PMM8/ks5/9LADf//73+dGPfsTznvc87r33Xp7znOfwnve8h/POO4/vfOc7/PjHP2bZsmX81m/9FpdccgkbN26cl/2a7SnFX8jMrRHxTOCmiPj3+szMzIiY2wnVGZRAdxHAs5/97PnYpCRJepp44QtfyKOPPsoJJ5zA8ccfz9ve9jZ+5Vd+hbVr1zI8PMypp546523+3u/9Hu9617tYu3Yt/f39fOpTn2LJkiWsX7+ez3zmMwwMDHDcccfxwQ9+kNtuu41LLrmEVqvFwMAAV1555bzsV8z1wrOI+BDwGPA7wCsz84FyyvBfM/N5EfE/S/nzZfnvAa/sDpn5u6W+Z7npDA8P58jIyJx3SpIkzd3dd9/N85///IVuxtPCdO9VRGzIzOHplt/vKcWIWB4RR3TLwGuBO4EbgO6dhuuAL5XyDcA7yt2KLwceKacevwK8NiKOLnc0vrbUSZIkHdJmc0pxNfDFiOgu/7nMvDEibgPWR8SFwH3Am8ryXwbOBTYDu4ELADJzR0T8BXBbWe7Dmblj3vZEkiQddu644w7e/va399QtWbKEW2+9dYFaNL39Bq7MvBd40TT124HXTFOfwMUzbOtq4Oq5N1OSJGlfa9euZdOmTQvdjP3y0T6SJGnCXK/tPhw9lffIwCVJkgAYGhpi+/bthq4nkZls376doaGhOa3nsxQlSRIAa9asYcuWLWzbtm2hm7KoDQ0NsWbNmjmtY+CSJEkADAwMcPLJJy90Mw5JnlKUJElqmIFLkiSpYQYuSZKkhhm4JEmSGmbgkiRJapiBS5IkqWEGLkmSpIYZuCRJkhpm4JIkSWqYgUuSJKlhBi5JkqSGGbgkSZIaZuCSJElqmIFLkiSpYQYuSZKkhhm4JEmSGmbgkiRJapiBS5IkqWEGLkmSpIYZuCRJkhpm4JIkSWqYgUuSJKlhBi5JkqSGGbgkSZIaZuCSJElqmIFLkiSpYQYuSZKkhhm4JEmSGmbgkiRJatisA1dE9EXEtyPin8r0yRFxa0RsjojrImKw1C8p05vL/JNq2/hAqf9eRJw93zsjSZK0GM2lh+u9wN216b8CrsjM5wI7gQtL/YXAzlJ/RVmOiHgB8GbghcA5wP+IiL4Da74kSdLiN6vAFRFrgF8G/r5MB/Bq4PqyyDXAG0r5vDJNmf+asvx5wLWZuSczfwBsBs6Yj52QJElazGbbw/XfgD8GOmX6WGBXZo6X6S3ACaV8AnA/QJn/SFl+on6adSZExEURMRIRI9u2bZvDrkiSJC1O+w1cEfF64KHM3HAQ2kNmXpWZw5k5vGrVqoPxkpIkSY3qn8UyPw/8akScCwwBK4G/BY6KiP7Si7UG2FqW3wqcCGyJiH7gSGB7rb6rvo4kSdIha789XJn5gcxck5knUV30/rXMfBtwM/CbZbF1wJdK+YYyTZn/tczMUv/mchfjycApwLfmbU8kSZIWqdn0cM3k/cC1EXE58G3gk6X+k8BnImIzsIMqpJGZd0XEeuC7wDhwcWa2D+D1JUmSnhai6nxanIaHh3NkZGShmyFJkrRfEbEhM4enm+cvzUuSJDXMwCVJktQwA5ckSVLDDFySJEkNM3BJkiQ1zMAlSZLUMAOXJElSwwxckiRJDTNwSZIkNczAJUmS1DADlyRJUsMMXJIkSQ0zcEmSJDXMwCVJktQwA5ckSVLDDFySJEkNM3BJkiQ1zMAlSZLUMAOXJElSwwxckiRJDTNwSZIkNczAJUmS1DADlyRJUsMMXJIkSQ0zcEmSJDXMwCVJktQwA5ckSVLDDFySJEkNM3BJkiQ1zMAlSZLUMAOXJElSwwxckiRJDdtv4IqIoYj4VkTcHhF3RcSfl/qTI+LWiNgcEddFxGCpX1KmN5f5J9W29YFS/72IOLupnZIkSVpMZtPDtQd4dWa+CDgdOCciXg78FXBFZj4X2AlcWJa/ENhZ6q8oyxERLwDeDLwQOAf4HxHRN587I0mStBjtN3Bl5bEyOVCGBF4NXF/qrwHeUMrnlWnK/NdERJT6azNzT2b+ANgMnDEveyFJkrSIzeoarojoi4hNwEPATcB/ALsyc7wssgU4oZRPAO4HKPMfAY6t10+zjiRJ0iFrVoErM9uZeTqwhqpX6tSmGhQRF0XESESMbNu2ramXkSRJOmjmdJdiZu4CbgZeARwVEf1l1hpgaylvBU4EKPOPBLbX66dZp/4aV2XmcGYOr1q1ai7NkyRJWpRmc5fiqog4qpSXAr8E3E0VvH6zLLYO+FIp31CmKfO/lplZ6t9c7mI8GTgF+NZ87YgkSdJi1b//RTgeuKbcUdgC1mfmP0XEd4FrI+Jy4NvAJ8vynwQ+ExGbgR1UdyaSmXdFxHrgu8A4cHFmtud3dyRJkhafqDqfFqfh4eEcGRlZ6GZIkiTtV0RsyMzh6eb5S/OSJEkNM3BJkiQ1zMAlSZLUMAOXJElSwwxckiRJDTNwSZIkNczAJUmS1DADlyRJUsMMXJIkSQ0zcEmSJDXMwCVJktQwA5ckSVLDDFySJEkNM3BJkiQ1zMAlSZLUsP6FbsBCGh1r857Pf5vlS/pZNtg3MV6xpJ9lg/0sX9JXjQf7WFbmDfX3MTTQYslANR7saxERC70rkiRpETusA9ee8Q4/2rGbx/eOs3tPm8f3jjM61pnTNloBS/r7WDrYx1B/i6GBPpYM9LF0oMWywX6GBqp5PdMDfSwdbLF0sJ8l3XX6W2XoY8lAi6Eynlo30BcGPEmSnmYO68B15NIBbvz9s3rq2p3sCWAT473j7N7bZnSswxNjbfaMtRkdq6ZHx9qMjtfKY5PL7Xh8L6NjbXbvbfPEWDXsHZ9bqKtrBQwN9FVDLeBVwa1V6ksvXBn3LjM5b+lAH8sGu4GwKg+VcRUO7b2TJGk+HNaBazp9rWDl0AArhwYae412JydC2J7xNnvGq6C2Z7zDnrEOo+Nt9ox1qnllXA923WUmA153mTbbH9vbs3x920/F0oFa713tlGq3p64Kc62JcjfALRuoQtvSwVrdYHVath70hvr7aLUMdZKkQ5uBawH0tYLlS/pZvuTgvf2Z2RPoukFt997xquet9MDt3ttbroLheE+46/biPfLEWE9dtwcvc25tG+yvAttEuCunZLuBbKCvxWB/q4yjGve1GOjW9UW1jcH+EvQmr7lbVgt63XKfAU+SdJAZuA4TETFxKvJImuu96wa73XuroPbE3nYpt3libJwn9nZKgCsBbW916nV0bN+g98RYm127xxhrdxhrJ3vHO6XcKeVkb3vuPXcDfVVoq4Z9y91wNzHdE/h6Q1593f6+Fv2tap3+vqC/FfS3qvJAmdffF7Qi6GsFfRG0WtVyrTLd16rmd9fvbr+/Lxgo2+qW7RmUpKcPA5fmVT3YHbN8sPHXy0zGO1XIe6L0zj1errfbPXVcrsebDG45EeC64W1svMN4p6rfM97hsT3jvQFvvMPeielqPN6ZY5fePOkrYa0e3rqBra9FT11PuOsGu1bQF9TWmRy62+2GyG7d1OmpQbFvShum1nXrI3qXmRyYXL5se+J1avtSf71W0NOO6G4jqhtMuuv3T9k3r0+UdDAZuPS0FhETPVErDuIp2rpOpwp9450qlI23O2W6Ko+1q3nj7aqu3Uk6WcZluXZW5e68at0q+HUDYH3bVdCrttmur59Ju8NEebJucui+djuZeM12VtusLzte2t3uTLanan9nYrr+GnM9lbzQWgH9rVZPeOwGvijhrxsEY6LMxHR/6WXsa0Ffq0XfRF03NFZ13eVjyvpRC5lBtY3+VtDXNzUc7htyIyAo65ZtVvvUrevuS2si1HZ7V/tLXV99HLXgWg/PEZP7E933aLJ+MvROvnetVn2/uvta9pve/ZcOJwYu6QC1WsFgKxg8zH9HOLMe/pgIbxPhr9R3ppSrsJZ0kp5gWA+h7RmCY3ediQCbTHm9yZA7sY1a8O1uswqunWr9st3MydfolEDZfZ0srzve89qTIXR8rMN4p92zXnebk9O98yaCbi3Utqe0/1DTDWJ9tVDb7aWsh7goy/Seni+n8nvK1bjbOzswZV5fKyZO/w9MBOvJ4Bq1dtTDYb1XuH+aywF6eo6jd73u9pkyHbX96y5fD6/1nt8oYXimbdfD/NTeX4Pt4mHgkjQvonvt2UI35BDVPX3eDWqZkNTDIVALc92gON7pDZ7dYNid1+50ql7RKT2indJb2p4SKrv19eU7nd7wOxkmqzZmrY3dkJnsG2q729mnnFQhtJ2MTdNzPNbuMDrWYbw9zljp9R3r9iq3OxPrTPbUVusfDuqhrrenshve6gGvhLdWb6CbDMKTlyvElIA5ERa7PcUxufxkfUz0+ramtGXqZQetqht3spcUJnp2u9OUYDw1gPdNCdtV73CL1SuX8OJnH71gx8LPRkl6GuiePtf86e1xrIfDyVA4ESxrvbfjnc7kuATQbl19vW7I7IbH7vbbnbLMlJ7aiZ7eiXWrbU9tZ8+2e16j3jNMzyn/+j7klGCePb3Pk0G43RN+a0F7Sk9vO6vQ289mk+YAAAWsSURBVO4JyZNtr/dI1y+naNfaPl1wT4D6cekeM3hKlzC88nmr+NQFZ8zjX9DcGLgkSYeliVOIGGSfrro9v1XvZ2eiB7Pq3azq2uU62GWDCxt5DFySJOlpafLGKVhK30I350kd3lf5SpIkHQQGLkmSpIYZuCRJkhpm4JIkSWqYgUuSJKlh+w1cEXFiRNwcEd+NiLsi4r2l/piIuCki7injo0t9RMTHI2JzRHwnIl5S29a6svw9EbGuud2SJElaPGbTwzUOvC8zXwC8HLg4Il4AXAp8NTNPAb5apgFeB5xShouAK6EKaMBlwMuAM4DLuiFNkiTpULbfwJWZD2TmxlJ+FLgbOAE4D7imLHYN8IZSPg/4dFa+CRwVEccDZwM3ZeaOzNwJ3AScM697I0mStAjN6RquiDgJeDFwK7A6Mx8os34CrC7lE4D7a6ttKXUz1UuSJB3SZh24ImIF8I/A72fmT+vzMrM8NvXARcRFETESESPbtm2bj01KkiQtqFk92iciBqjC1mcz8wul+sGIOD4zHyinDB8q9VuBE2urryl1W4FXTqn/16mvlZlXAVeV190WEffNem+eumcADx+E19HceWwWN4/P4uWxWdw8PovXgRybn5lpRuR+HrkdEUF1jdaOzPz9Wv3fANsz86MRcSlwTGb+cUT8MvBfgHOpLpD/eGaeUS6a3wB071rcCPxcZu54ijs1byJiJDOHF7od2pfHZnHz+CxeHpvFzeOzeDV1bGbTw/XzwNuBOyJiU6n7IPBRYH1EXAjcB7ypzPsyVdjaDOwGLgDIzB0R8RfAbWW5Dy+GsCVJktS0/QauzPw6EDPMfs00yydw8Qzbuhq4ei4NlCRJerrzl+YrVy10AzQjj83i5vFZvDw2i5vHZ/Fq5Njs9xouSZIkHRh7uCRJkhp2WAeuiDgnIr5Xnvt46f7XUJMi4uqIeCgi7qzVTfvMTh1cc32mqg6uiBiKiG9FxO3l+Px5qT85Im4tn3HXRcTgQrf1cBURfRHx7Yj4pzLtsVkkIuKHEXFHRGyKiJFSN++fbYdt4IqIPuDvqJ79+ALgLeUZkVo4n2Lfxz3N9MxOHVxzfaaqDq49wKsz80XA6cA5EfFy4K+AKzLzucBO4MIFbOPh7r1Uj8br8tgsLq/KzNNrPwcx759th23gonqA9ubMvDcz9wLXUj0HUgskM28Bpv5UyEzP7NRB9BSeqaqDqDy79rEyOVCGBF4NXF/qPT4LJCLWAL8M/H2ZDjw2i928f7YdzoHLZzs+Pcz0zE4tkFk+U1UHWTlltYnqqR83Af8B7MrM8bKIn3EL578Bfwx0yvSxeGwWkwT+JSI2RMRFpW7eP9tm9WgfaTHIzIwIb6tdQFOfqVp9Ua94fBZWZraB0yPiKOCLwKkL3CQBEfF64KHM3BARr1zo9mhav5CZWyPimcBNEfHv9Znz9dl2OPdwzfTMRy0uD5ZndTLlmZ06yJ7smaplvsdnEcjMXcDNwCuAoyKi+8Xaz7iF8fPAr0bED6kuXXk18Ld4bBaNzNxaxg9RfVk5gwY+2w7nwHUbcEq5U2QQeDNwwwK3Sfu6AVhXyuuALy1gWw5b5ZqTTwJ3Z+bHarM8PotARKwqPVtExFLgl6ius7sZ+M2ymMdnAWTmBzJzTWaeRPX/zNcy8214bBaFiFgeEUd0y8BrgTtp4LPtsP7h04g4l+rceh9wdWZ+ZIGbdFiLiM8Dr6R6UvuDwGXA/wbWA8+mPLPTZ3AefBHxC8C/AXcweR3KB6mu4/L4LLCI+E9UF/b2UX2RXp+ZH46I51D1qhwDfBv4rczcs3AtPbyVU4p/lJmv99gsDuU4fLFM9gOfy8yPRMSxzPNn22EduCRJkg6Gw/mUoiRJ0kFh4JIkSWqYgUuSJKlhBi5JkqSGGbgkSZIaZuCSJElqmIFLkiSpYQYuSZKkhv1/5i9vz1+/PKwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss_df = pd.DataFrame(history.history)\n",
    "\n",
    "# Plot loss vs epochs\n",
    "\n",
    "loss_df.plot(figsize=(10,5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "q2co33ICNWKo",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "q2co33ICNWKo",
    "outputId": "88398532-4ffd-45a7-e38f-3dcfb7ac66e9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5/5 [==============================] - 0s 2ms/step - loss: 7934.7148\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "7934.71484375"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(test_data_prepared, test_label)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "nyhlBiJeGDoS",
   "metadata": {
    "id": "nyhlBiJeGDoS"
   },
   "source": [
    "The results are not impressive. Let's try to think why the model is not doing well. There are some few things to draw from the graph above. \n",
    "\n",
    "* Ideally, both validation and training loss should decrease during training. If training doesn't decrease, it's very likely that the input features don't contain enough information to predict the output. \n",
    "\n",
    "* Both `loss` and `val_loss` didn't improve alot, and there is no evidence that training for more epochs will improve the results. Quite the opposite, there is evidence that it will not improve.\n",
    "\n",
    "* How about adding more layers, or neurons? There is a notion that a model is as good as the data it was trained on. In most cases, the sure thing to improve or add more data.\n",
    "\n",
    "Let's see what we can improve."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "vh8WP3z-VwnI",
   "metadata": {
    "id": "vh8WP3z-VwnI"
   },
   "source": [
    "<a name='3-6'></a>\n",
    "\n",
    "## 3.6 Improving the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "EPwS-esEV7ii",
   "metadata": {
    "id": "EPwS-esEV7ii"
   },
   "source": [
    "This data is very skewed. The burned area of the forest varies from 0.00 to 1090.84 but it's skewed to ward 0. Take a look at it again below..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "ILd6CEsmWutk",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 296
    },
    "id": "ILd6CEsmWutk",
    "outputId": "6d46b91f-fa29-4668-c769-8f9df66af2b4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f8ce932f5d0>"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEGCAYAAABy53LJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAewklEQVR4nO3de5RdZZ3m8e+TqlzIrUJCCAlBEkyUjigqMeDQzjDtgNitBrthDEqDM47YIyxaW8cFdkurS9dqp73hknZJiy0yo6C0aJqmxeaia3khUBE0XMxQJEAKQ0hCksrFXCr5zR/v3tZJUUmdc2rvOrWrns9ae+1z9u28Z6+TevK+e+/3VURgZmZWr3GtLoCZmVWLg8PMzBri4DAzs4Y4OMzMrCEODjMza0h7qwswHI477rhYsGBBq4thZlYpq1ev3hIRs/svHxPBsWDBAjo7O1tdDDOzSpH09EDL3VRlZmYNcXCYmVlDHBxmZtYQB4eZmTXEwWFmZg1xcJiZWUMcHGZm1hAHR9ncbb2ZjTIOjjIdPAiLFsGXvtTqkpiZFcbBUabubli3Dv72b2H79laXxsysEA6OMq1bl+bbt8NnP9vaspiZFcTBUaY8OM46C667Dg4caG15zMwK4OAo07p10N4Ol1wCu3bB88+3ukRmZkPm4CjTunVw8skwf356v2lTa8tjZlYAB0eZ1q2DU06BE05I7597rrXlMTMrgIOjTA4OMxuFHBxl6emBLVtg4UKYMyctc3CY2Sjg4CjL+vVpfsopMGkSzJjh4DCzUcHBUZb8VtxTTknzE05wcJjZqODgKEv/4Jgzx8FhZqOCg6Ms3d0wZQoce2x67xqHmY0SDo6ybNsGM2f2vXdwmNko4eAoy/bt6YJ47oQTYOdO2L27dWUyMyuAg6Ms27b1NVNB37McfnrczCrOwVGWgWoc4OYqM6s8B0dZHBxmNko5OMpypKYqB4eZVZyDowy9velCeG2NY/ZsGDcONm5sXbnMzArg4ChDT0+a19Y42trS+xdeaE2ZzMwK4uAow7ZtaV5b4wDo6IAdO4a/PGZmBXJwlGH79jR3cJjZKOTgKENe46htqoIUHHmomJlVlIOjDK5xmNkoVmpwSDpf0lpJXZKuHmD9REm3ZutXSVqQLT9X0mpJa7L5H9Xsc0a2vEvSlySpzO/QlDw4BqpxODjMrOJKCw5JbcD1wJuBJcDFkpb02+w9wLaIWAR8AfhMtnwL8NaIeCVwGXBzzT5fAd4LLM6m88v6Dk3zxXEzG8XKrHEsA7oiYl1E7AduAZb322Y5cFP2+jbgjZIUEQ9FxG+z5Y8Cx2S1k7nA9Ii4PyIC+CZwQYnfoTnbt6fbb6dOPXx5R0e6VffQodaUy8ysAGUGx4nAhpr33dmyAbeJiF5gBzCr3zZ/BvwyIvZl23cPckwAJF0uqVNS5+bNm5v+Ek3Juxvp34rW0QERsGvX8JbHzKxAI/riuKRXkJqv3tfovhFxQ0QsjYils2fPLr5wR7Nt24ubqSAFB7i5yswqrczgeBY4qeb9/GzZgNtIagc6gK3Z+/nA7cClEfFkzfbzBzlm6/Xv4DDn4DCzUaDM4HgQWCxpoaQJwApgZb9tVpIufgNcCNwbESFpBvCvwNUR8bN844jYCPRIOiu7m+pS4Aclfofm9O/gMOfgMLNRoLTgyK5ZXAncBTwOfCciHpX0SUlvyza7EZglqQv4KyC/ZfdKYBFwraSHs+n4bN37ga8BXcCTwL+V9R2a5hqHmY1i7WUePCLuBO7st+zamtd7gYsG2O9TwKeOcMxO4LRiS1qw7dtd4zCzUWtEXxyvLF8cN7NRzMFRtL17Yd8+B4eZjVoOjqLloTB9+ovXTZ6cHgx0cJhZhTk4irZ7d5pPm/bidZK7HTGzynNwFC1/Krx/dyO5GTMcHGZWaQ6Oog0WHK5xmFnFOTiK5uAws1HOwVE0B4eZjXIOjqI5OMxslHNwFM3BYWajnIOjaHlwTJky8Pp8MKeI4SuTmVmBHBxFy5/jmDx54PUdHWkEQA/mZGYV5eAo2q5dqbYx7ginNn+ivKdn+MpkZlYgB0fRdu068vUN6HuifOfO4SmPmVnBHBxFc3CY2Sjn4CjaYMGRr/M1DjOrKAdH0VzjMLNRzsFRNAeHmY1yDo6i5XdVHYmDw8wqzsFRtN27XeMws1HNwVG0wZqqpkxJAzo5OMysohwcRRssOKS03sFhZhXl4CjSgQOwb9/RgwNSc5VvxzWzinJwFCnvp2qw4HCNw8wqzMFRpMG6VM9Nm+bgMLPKcnAUycFhZmOAg6NIDg4zGwMcHEXKr3Ec7QFAcHCYWaU5OIrkGoeZjQEOjiI1Ehy+HdfMKsrBUaR6g2PqVNizBw4eLL9MZmYFc3AUqZEaR+32ZmYV4uAoUqPB4escZlZBDo4i7doF48fDhAlH387BYWYV5uAo0mAdHOYcHGZWYQ6OIg02iFPOwWFmFVZqcEg6X9JaSV2Srh5g/URJt2brV0lakC2fJek+SbskfbnfPj/OjvlwNh1f5ndoyGCDOOV8cdzMKqy9rANLagOuB84FuoEHJa2MiMdqNnsPsC0iFklaAXwGeAewF/gYcFo29feuiOgsq+xNq7epKt/GNQ4zq6AyaxzLgK6IWBcR+4FbgOX9tlkO3JS9vg14oyRFxO6I+CkpQKrD1zjMbAwoMzhOBDbUvO/Olg24TUT0AjuAWXUc+5+yZqqPSdJAG0i6XFKnpM7Nmzc3XvpmODjMbAyo4sXxd0XEK4E3ZNOfD7RRRNwQEUsjYuns2bOHp2T1BsfkyTBunIPDzCqpzOB4Fjip5v38bNmA20hqBzqArUc7aEQ8m813At8iNYmNDPUGh8cdN7MKKzM4HgQWS1ooaQKwAljZb5uVwGXZ6wuBeyMijnRASe2SjstejwfeAjxSeMmbVW9wgHvINbPKKu2uqojolXQlcBfQBnw9Ih6V9EmgMyJWAjcCN0vqAl4ghQsAkp4CpgMTJF0AnAc8DdyVhUYbcDfwj2V9h4ZENB4cvh3XzCqotOAAiIg7gTv7Lbu25vVe4KIj7LvgCIc9o6jyFWrv3hQe9TwACG6qMrPKquLF8ZGp3g4Oc26qMrOKcnAUxcFhZmOEg6MoDg4zGyMcHEVxcJjZGFFXcEj6nqQ/keSgORIHh5mNEfUGwT8A7wSekPR3kl5eYpmqqZng2LsXenvLK5OZWQnqCo6IuDsi3gW8FngKuFvSzyX9t+yZCms0OPLt/CyHmVVM3U1PkmYB7wb+B/AQcB0pSP69lJJVze7daV7vcxzu6NDMKqquBwAl3Q68HLgZeGtEbMxW3Spp5I2L0QrNNFWBg8PMKqfeJ8f/MXsK/PckTYyIfRGxtIRyVU8eHJMn17e9g8PMKqrepqpPDbDsF0UWpPJ27Uqh0dZW3/YODjOrqKPWOCSdQBps6RhJrwHyQZOmA3X+13qMaKSDQ3BwmFllDdZU9SbSBfH5wOdrlu8EPlpSmarJwWFmY8RRgyMibgJukvRnEfHPw1Smamo0OHw7rplV1GBNVZdExP8BFkj6q/7rI+LzA+w2NrnGYWZjxGBNVflDCQ38RRyjdu/uC4N6HHOMxx03s0oarKnqq9n8E8NTnArbtQvmzq1/e8n9VZlZJdXbyeH/ljRd0nhJ90jaLOmSsgtXKY02VYGDw8wqqd7nOM6LiB7gLaS+qhYB/6usQlWSg8PMxoh6gyNv0voT4LsRsaOk8lSXg8PMxoh6uxy5Q9JvgN8B/1PSbGBvecWqmN7e1EV6M8Hh23HNrGLq7Vb9auA/AEsj4gCwG1heZsEqJe8Zt9HgmDrVNQ4zq5x6axwAp5Ke56jd55sFl6eaGu0ZN+emKjOroHq7Vb8ZeCnwMHAwWxw4OJI8OOodiyPn4DCzCqq3xrEUWBIRUWZhKisPjkYeAMy3d3CYWcXUe1fVI8AJZRak0vI//s00Ve3bBwcOFF8mM7OS1FvjOA54TNIDwL58YUS8rZRSVc1QahyQgmfmzGLLZGZWknqD4+NlFqLyhnJxHBwcZlYpdQVHRPxE0snA4oi4W9JkoM6h7saAvKmq0RrH9Olp3tNTbHnMzEpUb19V7wVuA76aLToR+H5ZhaqcZmscDg4zq6B6L45fAZwN9ABExBPA8WUVqnKaDY6OjjTf4R5czKw66g2OfRGxP3+TPQToW3NzO3fCpEnQ3sjzlDg4zKyS6g2On0j6KHCMpHOB7wL/Ul6xKqaZDg7BwWFmlVRvcFwNbAbWAO8D7gT+pqxCVY6Dw8zGkHrvqjok6fvA9yNic8llqp6dOxu/owrS8LFtbb44bmaVctQah5KPS9oCrAXWZqP/XTs8xauIZmscUqp1uMZhZhUyWFPVB0l3U70uImZGxEzgTOBsSR8c7OCSzpe0VlKXpKsHWD9R0q3Z+lWSFmTLZ0m6T9IuSV/ut88ZktZk+3xJkur8ruVptsYBDg4zq5zBguPPgYsjYn2+ICLWAZcAlx5tR0ltwPXAm4ElwMWSlvTb7D3AtohYBHwB+Ey2fC/wMeDDAxz6K8B7gcXZdP4g36F8zdY4wMFhZpUzWHCMj4gt/Rdm1znGD7LvMqArItZlt/LewosHf1oO3JS9vg14oyRFxO6I+Cn9RhmUNBeYHhH3Zz31fhO4YJBylM/BYWZjyGDBsb/JdZCeLt9Q8747WzbgNhHRC+wAZg1yzO5BjgmApMsldUrq3Ly55Ov5Q2mqmj7dF8fNrFIGC47TJfUMMO0EXjkcBWxWRNwQEUsjYuns2bPL/TDXOMxsDDnq7bgRMZSODJ8FTqp5Pz9bNtA23dnT6B3A1kGOOX+QYw6v/fvT5IvjZjZG1PsAYDMeBBZLWihpArACWNlvm5XAZdnrC4F7jzbKYERsBHoknZXdTXUp8IPii96AZvupyuXB4cEVzawiGuxcqX4R0SvpSuAuUhfsX4+IRyV9EuiMiJXAjcDNkrqAF0jhAoCkp4DpwARJFwDnRcRjwPuBbwDHAP+WTa0z1OCYPh0OHoQ9exofs9zMrAVKCw6AiLiT1D1J7bJra17vBS46wr4LjrC8EzituFIOUbOj/+Xybkd6ehwcZlYJZTZVjQ3Njjeec39VZlYxDo6hKuIaBzg4zKwyHBxDVVRTlYPDzCrCwTFUQ22qyoePdXCYWUU4OIaqyIvjZmYV4OAYKl8cN7MxxsExVHmNY/Lk5vafNi2Ny+HgMLOKcHAMVd5P1bgmT+W4cSk8HBxmVhEOjqHaubP5Zqrc9OkODjOrDAfHUA2lZ9zcjBmwfXsx5TEzK5mDY6h27Oi7wN2smTPhhReKKY+ZWckcHENVRHDMmuXgMLPKcHAMVVHBsfVow5CYmY0cDo6hKqqpautWj8lhZpXg4Biqomoc+/enMTnMzEY4B8dQHDqUbsctIjjAzVVmVgkOjqHYuTM1LxXRVAUODjOrBAfHUOTPXhRV4/CdVWZWAQ6Oocif9nZTlZmNIQ6OoSgqOPKmKtc4zKwCHBxDkQfHjBlDO46vcZhZhTg4hqKoGsfEiTBlioPDzCrBwTEURQUHuNsRM6sMB8dQFB0crnGYWQU4OIZixw6YMAEmTRr6sfJuR8zMRjgHx1AU0d1Izk1VZlYRDo6hKDo4XOMwswpwcAxFkcGRD+Z06FAxxzMzK4mDYyi2by+2xnHoEPT0FHM8M7OSODiGouimKnBzlZmNeA6OoSgjOLZsKeZ4ZmYlcXAMRZHBMXdumm/cWMzxzMxK4uBo1sGDxQzilJs3L80dHGY2wjk4mrVzZ5oXFRyzZ8O4cfDb3xZzPDOzkjg4mlVUz7i5tjaYM8c1DjMb8RwczSqyn6rcvHmucZjZiFdqcEg6X9JaSV2Srh5g/URJt2brV0laULPummz5Wklvqln+lKQ1kh6W1Flm+Y8qv/spvxuqCHPnusZhZiNeacEhqQ24HngzsAS4WNKSfpu9B9gWEYuALwCfyfZdAqwAXgGcD/xDdrzcf46IV0fE0rLKP6g8OI47rrhjOjjMrALKrHEsA7oiYl1E7AduAZb322Y5cFP2+jbgjZKULb8lIvZFxHqgKzveyJEHx+zZxR1z3jx4/nno7S3umGZmBSszOE4ENtS8786WDbhNRPQCO4BZg+wbwI8krZZ0+ZE+XNLlkjoldW7evHlIX2RA+THzYV+LMHcuRMCmTcUd08ysYFW8OP6HEfFaUhPYFZL+40AbRcQNEbE0IpbOLrJWkNuyBY49FsaPL+6Y+bMcvkBuZiNYmcHxLHBSzfv52bIBt5HUDnQAW4+2b0Tk8+eB22lVE9bmzcVe3wA/PW5mlVBmcDwILJa0UNIE0sXulf22WQlclr2+ELg3IiJbviK762ohsBh4QNIUSdMAJE0BzgMeKfE7HNmWLcVe3wAHh5lVQntZB46IXklXAncBbcDXI+JRSZ8EOiNiJXAjcLOkLuAFUriQbfcd4DGgF7giIg5KmgPcnq6f0w58KyJ+WNZ3OKotW+Dkk4s95pw5ILmpysxGtNKCAyAi7gTu7Lfs2prXe4GLjrDvp4FP91u2Dji9+JI2YfNmOOOMYo/Z3g7HH+8ah5mNaFW8ON56EeU0VUG6QP5s/0tBZmYjh4OjGTt3wv79xV8cB1iwANavL/64ZmYFcXA0o4yH/3KLFsGTT6Zu283MRiAHRzPK6G4kt3hxqs10dxd/bDOzAjg4mpE/NV5GcCxalOZdXcUf28ysAA6OZpTZVLV4cZo/8UTxxzYzK4CDoxll1jjmzYNJk1zjMLMRy8HRjC1bYMIEmDat+GOPG5eaq1zjMLMRysHRjC1bUm0jPcFevEWLXOMwsxHLwdGMMjo4rLV4cbol99Ch8j7DzKxJDo5mdHfDif2HFinQokWwbx9s2DD4tmZmw8zB0YxnnoGXvKS847/sZWn+m9+U9xlmZk1ycDRq9+5yesat9ZrXpHlnZ3mfYWbWJAdHo/LmozJrHB0dcOqp8MAD5X2GmVmTHByNevrpNC+zxgGwbBmsWpV64jUzG0EcHI165pk0L7PGASk4Nm1yn1VmNuI4OBr19NPQ1pae8C7TsmwodTdXmdkI4+Bo1DPPpFtx20sdPBFe9ar0dLqDw8xGGAdHo55+uvzrGwATJ8KrXw0//3n5n2Vm1gAHR6PKfoaj1rnnwi9+AVu3Ds/nmZnVwcHRiIMH08Xq4ahxAFxwQfrMO+4Yns8zM6uDg6MRGzdCb+/w1TjOOAPmz4fbbx+ezzMzq4ODoxFPPZXmw1XjkFKt40c/gj17huczzcwG4eBoxK9+leannTZ8n/n2t8Pvfgc/+MHwfaaZ2VE4OBqxenUaLrbMnnH7O+ec1Onh5z7np8jNbERwcDTil79M1x3KGsBpIOPGwYc+lELrxz8evs81MzsCB0e99u6FRx+F1752+D/70kvh+OPh0592rcPMWs7BUa9HHkl3VLUiOCZNgo9+FO65B7797eH/fDOzGg6Oev3yl2neiuAAuPJKOOssuOqq1PmhmVmLODjqtXo1zJgBCxa05vPb2uDGG9NAUm99K+za1ZpymNmY5+Co109/OvwXxvtbsgS+851U+3nb22DbttaVxczGLAdHPdasgccegz/901aXJNU2vvGNFGSve517zzWzYefgqMe3vpWaii66qNUlSS65BO67Lz0YeOaZ8O53w+OPt7pUZjZGODgGc+hQupPpvPPSw38jxdlnp7D48Ifh1ltTM9Yb3gBf/GLfKIVmZiVwcAzm7rvTGBzvfGerS/Ji06fD3/99CopPfQp6euCDH0x9ab3iFfC+98HNN0NXl5//MLPCKMbAH5SlS5dGZ2dn4zv29KSR+Nrb4de/hsmTiy9c0Z54IvWm+5OfwM9+Bjt2pOWzZqXhaJctS81by5alZcMlIpVl9+50PtvbYerUNGCVmY1IklZHxNL+y0sd/1TS+cB1QBvwtYj4u37rJwLfBM4AtgLviIinsnXXAO8BDgJXRcRd9RyzUB/4AGzYkC5EVyE0ABYvho98JE2HDqWn3e+/H1atStMPf9hX+3jpS1OInHkmLF0Kp58OU6Y09nkRaaCp3/42dTtfO/Vftnfv4ftKaez2U06BRYtSSL/ylWk+kpoFzewwpdU4JLUB/w84F+gGHgQujojHarZ5P/CqiPgLSSuAt0fEOyQtAb4NLAPmAXcDL8t2O+oxB9JUjePAAVixAk49NXX1MVrs3JmeScmDZNWq9Ace0h/yl70M/uAPYM6cNE2fngaTOnAg1RY2bTp8eu65tK6/jg6YO7dvmjcvzadOTcfr7YUXXoD162HdOli7Fp5/vm//OXNSgORh8pKXpBrSrFkpxNva+iYp3SiwZ0/f1NMD27enadu2vte13dOPH5+ezcmnY49N846OFKD5NHlyGv+9lbdij1VH+vt0tL9bRe5TRhn27En/DvOpp+fwaceONN+9O/2+J0xI09SpMG1a+jc5bdrhr2uXFViLb0WNYxnQFRHrsgLcAiwHav/ILwc+nr2+DfiyJGXLb4mIfcB6SV3Z8ajjmMUYPx5uuy39r300mTYt9bh7zjl9y7q707MhDz2UpieeSLWsrVsP/wfQ1tYXKHPmpOsoteGQB8QJJzRXQ9u0Kd36vGZNahpcswauv/7FNZVm5SGQB8C+fekfaCP/eZJSx5MDzY8ULMP5R2k49injeHa4adP6/pO1f3/6rf7ud/XtO25c33+qxo2DLVvgmGMKLV6ZwXEisKHmfTdw5pG2iYheSTuAWdny+/vtm/dlPtgxAZB0OXB59naXpLVNfIciHAdsadFnF+vgwVQ7yWsoxRmec7R7d5qGIiKdh+E3en5H5Rk95yivjTTj0KE05S0Bh/8nrtFzNOCodaVe42iliLgBuKHV5ZDUOVBVz/r4HA3O52hwPkeDK+oclXk77rPASTXv52fLBtxGUjvQQbpIfqR96zmmmZmVqMzgeBBYLGmhpAnACmBlv21WApdlry8E7o10tX4lsELSREkLgcXAA3Ue08zMSlRaU1V2zeJK4C7SrbNfj4hHJX0S6IyIlcCNwM3Zxe8XSEFAtt13SBe9e4ErIuIgwEDHLOs7FKTlzWUV4HM0OJ+jwfkcDa6QczQmHgA0M7PiuMsRMzNriIPDzMwa4uAokaTzJa2V1CXp6laXpxUknSTpPkmPSXpU0l9my2dK+ndJT2TzY7PlkvSl7Jz9WlKLxuodfpLaJD0k6Y7s/UJJq7JzcWt2QwjZTSO3ZstXSVrQynIPF0kzJN0m6TeSHpf0ev+ODifpg9m/s0ckfVvSpDJ+Rw6OkmRdrlwPvBlYAlycdaUy1vQCH4qIJcBZwBXZebgauCciFgP3ZO8hna/F2XQ58JXhL3LL/CVQO7DKZ4AvRMQiYBup7zay+bZs+Rey7caC64AfRsSpwOmkc+XfUUbSicBVwNKIOI10A9EKyvgdRYSnEibg9cBdNe+vAa5pdblaPQE/IPU1thaYmy2bC6zNXn+V1P9Yvv3vtxvNE+mZpHuAPwLuAER6wre9/++JdFfh67PX7dl2avV3KPn8dADr+39P/44OOxd5Txwzs9/FHcCbyvgducZRnoG6XDnxCNuOCVlV+DXAKmBORGzMVj0HzMlej9Xz9kXgI0DeOdosYHtE9Gbva8/DYV31AHlXPaPZQmAz8E9Zc97XJE3Bv6Pfi4hngc8CzwAbSb+L1ZTwO3Jw2LCQNBX4Z+ADEdFTuy7Sf3nG7H3hkt4CPB8Rq1tdlhGsHXgt8JWIeA2wm75mKcC/o+z6znJSyM4DpgDnl/FZDo7yuHuUjKTxpND4vxHxvWzxJklzs/VzgbxP9bF43s4G3ibpKeAWUnPVdcCMrCseOPw8HKmrntGsG+iOiFXZ+9tIQeLfUZ//AqyPiM0RcQD4Hum3VfjvyMFRHnePQrq7hdRDwOMR8fmaVbXdzVxGuvaRL780uyvmLGBHTVPEqBQR10TE/IhYQPqd3BsR7wLuI3XFAy8+RwN11TNqRcRzwAZJL88WvZHUs4R/R32eAc6SNDn7d5efo+J/R62+oDOaJ+CPSQNPPQn8davL06Jz8Iek5oNfAw9n0x+T2lLvAZ4gDdQ1M9tepLvRngTWkO4Qafn3GMbzdQ5wR/b6FFIfbV3Ad4GJ2fJJ2fuubP0prS73MJ2bVwOd2W/p+8Cx/h296Bx9AvgN8AhwMzCxjN+RuxwxM7OGuKnKzMwa4uAwM7OGODjMzKwhDg4zM2uIg8PMzBri4DAzs4Y4OMxaJOtB2axyHBxmJZH0fUmrs/ERLs+W7ZL0OUm/Al4v6RJJD0h6WNJX8zCR9BVJndm+n2jpFzHrx8FhVp7/HhFnAEuBqyTNInU8tyoiTif1C/QO4OyIeDVwEHhXtu9fR8RS4FXAf5L0quEvvtnA2gffxMyadJWkt2evTyINKnSQ1OEjpL6EzgAeTF0LcQx9nfT916yW0k4aZ2IJqasNs5ZzcJiVQNI5pN5KXx8ReyT9mNQ30N6IOJhvBtwUEdf023ch8GHgdRGxTdI3sn3NRgQ3VZmVo4M0LOceSaeShs3t7x7gQknHw+/HYT8ZmE4ab2KHpDmkYVDNRgzXOMzK8UPgLyQ9Thq29P7+G0TEY5L+BviRpHHAAeCKiLhf0kOkXk43AD8bxnKbDcq945qZWUPcVGVmZg1xcJiZWUMcHGZm1hAHh5mZNcTBYWZmDXFwmJlZQxwcZmbWkP8PoXs95aJBMicAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.kdeplot(data=train_data, x='area', color='red')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "nIwKjBaHWxJ3",
   "metadata": {
    "id": "nIwKjBaHWxJ3"
   },
   "source": [
    "As the data source suggested, it may make sense to model it with logarithmic loss. Let's use `LogCosh class` that is available in [Keras Regression losses](https://keras.io/api/losses/regression_losses/#logcosh-class). That type of loss function computes the logarithm of the hyperbolic cosine of the prediction error.\n",
    "\n",
    "Also, as the most values of the target label `area` falls between 0 and 1, we can use the `sigmoid` activation function so that the output of the network doesn't swing above such range. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "oigmddtYZFqn",
   "metadata": {
    "id": "oigmddtYZFqn"
   },
   "outputs": [],
   "source": [
    "model_2 = keras.models.Sequential([\n",
    "                                 \n",
    "            # The first layers must specify the input shape always\n",
    "            keras.layers.Dense(12, activation='relu', input_shape=input_shape),\n",
    "            keras.layers.Dense(6, activation='relu'),\n",
    "\n",
    "            # The last layer usually doesn't have activation function in regression\n",
    "            keras.layers.Dense(1, activation='sigmoid')                \n",
    "\n",
    "])\n",
    "\n",
    "# Now we compile the model\n",
    "model_2.compile(loss='log_cosh', optimizer='adam')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "HyNdo_NtZRX_",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "HyNdo_NtZRX_",
    "outputId": "fe41f3af-3069-4045-e0bf-534366abbafc"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "12/12 [==============================] - 1s 13ms/step - loss: 11.5257 - val_loss: 14.5699\n",
      "Epoch 2/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 11.5059 - val_loss: 14.5482\n",
      "Epoch 3/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 11.4805 - val_loss: 14.5220\n",
      "Epoch 4/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.4523 - val_loss: 14.4939\n",
      "Epoch 5/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.4251 - val_loss: 14.4703\n",
      "Epoch 6/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.4017 - val_loss: 14.4518\n",
      "Epoch 7/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3838 - val_loss: 14.4395\n",
      "Epoch 8/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 11.3710 - val_loss: 14.4299\n",
      "Epoch 9/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3635 - val_loss: 14.4229\n",
      "Epoch 10/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3571 - val_loss: 14.4192\n",
      "Epoch 11/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 11.3530 - val_loss: 14.4164\n",
      "Epoch 12/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3500 - val_loss: 14.4147\n",
      "Epoch 13/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 11.3472 - val_loss: 14.4145\n",
      "Epoch 14/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 11.3447 - val_loss: 14.4134\n",
      "Epoch 15/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 11.3421 - val_loss: 14.4125\n",
      "Epoch 16/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3404 - val_loss: 14.4112\n",
      "Epoch 17/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3383 - val_loss: 14.4111\n",
      "Epoch 18/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 11.3368 - val_loss: 14.4110\n",
      "Epoch 19/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3355 - val_loss: 14.4111\n",
      "Epoch 20/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 11.3342 - val_loss: 14.4114\n",
      "Epoch 21/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3332 - val_loss: 14.4112\n",
      "Epoch 22/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 11.3320 - val_loss: 14.4116\n",
      "Epoch 23/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 11.3310 - val_loss: 14.4113\n",
      "Epoch 24/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3302 - val_loss: 14.4119\n",
      "Epoch 25/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3293 - val_loss: 14.4125\n",
      "Epoch 26/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 11.3287 - val_loss: 14.4121\n",
      "Epoch 27/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3279 - val_loss: 14.4134\n",
      "Epoch 28/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3271 - val_loss: 14.4139\n",
      "Epoch 29/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 11.3264 - val_loss: 14.4145\n",
      "Epoch 30/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3256 - val_loss: 14.4154\n",
      "Epoch 31/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 11.3251 - val_loss: 14.4155\n",
      "Epoch 32/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3243 - val_loss: 14.4163\n",
      "Epoch 33/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 11.3238 - val_loss: 14.4172\n",
      "Epoch 34/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3233 - val_loss: 14.4174\n",
      "Epoch 35/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3228 - val_loss: 14.4190\n",
      "Epoch 36/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3223 - val_loss: 14.4194\n",
      "Epoch 37/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3218 - val_loss: 14.4196\n",
      "Epoch 38/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3213 - val_loss: 14.4186\n",
      "Epoch 39/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3208 - val_loss: 14.4175\n",
      "Epoch 40/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 11.3203 - val_loss: 14.4177\n",
      "Epoch 41/50\n",
      "12/12 [==============================] - 0s 3ms/step - loss: 11.3197 - val_loss: 14.4182\n",
      "Epoch 42/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 11.3193 - val_loss: 14.4187\n",
      "Epoch 43/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 11.3188 - val_loss: 14.4193\n",
      "Epoch 44/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3183 - val_loss: 14.4207\n",
      "Epoch 45/50\n",
      "12/12 [==============================] - 0s 6ms/step - loss: 11.3178 - val_loss: 14.4205\n",
      "Epoch 46/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3172 - val_loss: 14.4200\n",
      "Epoch 47/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3168 - val_loss: 14.4202\n",
      "Epoch 48/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 11.3164 - val_loss: 14.4203\n",
      "Epoch 49/50\n",
      "12/12 [==============================] - 0s 5ms/step - loss: 11.3162 - val_loss: 14.4223\n",
      "Epoch 50/50\n",
      "12/12 [==============================] - 0s 4ms/step - loss: 11.3158 - val_loss: 14.4234\n"
     ]
    }
   ],
   "source": [
    "history = model_2.fit(training_data_prepared, train_label, \n",
    "                    validation_data = (test_data_prepared, test_label), epochs=50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "uXOqmAiMfHOh",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 337
    },
    "id": "uXOqmAiMfHOh",
    "outputId": "0078b834-e792-42a3-8b62-f98fef981d62"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7f8c9778b290>"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAAEvCAYAAAB2a9QGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dfXRcdb3v8c93Mnlom/Q5bdqmpa2WFmigeCNHl4LAXaciF+GqRzmICh6xS1R8vCqCHjgIS489S85dV5fIwlr0Vg69Kh6uqMhRjpV1UUlrS1seKtYWkkKTPlD6QJ5mvvePvSeZTGaSNMmvM0ner7Vm7b1/+7d/+zuzy8xn9p5szN0FAACA0ZUodgEAAADjESELAAAgAEIWAABAAIQsAACAAAhZAAAAARCyAAAAAkgWu4B8Zs+e7YsXLy52GQAAAIPavHnzAXevzW0vyZC1ePFiNTU1FbsMAACAQZnZ3nztXC4EAAAIgJAFAAAQACELAAAgAEIWAABAAIQsAACAAAhZAAAAARCyAAAAAiBkAQAABEDIAgAACKAk7/ge3M4HJCuTapdLM5dKZeXFrggAAIwzEzNk/eZ26eBz0XwiGQWt2adHj9rlvfOV1cWtEwAAjFkTM2St+U/pwJ+lA7uktmej6YFd0q5fSunu3n5T66PQVdcgzTtbmrdKmrFESnCVFQAADGzQkGVm6yRdJqnV3VfmrPuspH+RVOvuB/Jsm5K0PV583t0vH3nJo6CyRlrwuuiRLdUlHdrdN3y1PiU9/i0p3RX1qaiR6lZKdWfHwescqXYFlxwBAEAfQzmTtV7SNyV9P7vRzBZKWi3p+QG2fdXdVw27ulOtrDw6c1W7XDrj7b3t3Z1S29PSi09KLz0ZTf/0v6U/Ho+3q5DmnCHNPzd+vC5aJngBADBhDRqy3H2TmS3Os+pOSZ+X9O+jXFPpSVZEZ6zmndPblk5FZ71e3BYHr23Szp9Km9fH21RFlxnnx2fM5p8rzVrGpUYAACaIYf0my8yukNTi7tvMbKCuVWbWJKlb0tfc/afD2V9JSpRJs5dFj4a/i9rco+C170/Ro2VLfMbrO9H6ihpp/qroUuPcM6OzXbUrpIopxXseAAAgiJMOWWY2WdJNii4VDuY0d28xs6WSfmNm2939LwXGXSNpjSQtWrToZMsqDWbSrNdEj0zwSqei33ft29IbvJq+K3W3ZzaSZiyW5p4Vha45Z0aPWa/hciMAAGOYufvgnaLLhT9z95Vm1iDp15JOxKvrJe2TdJ67vzTAGOvjMX402P4aGxu9qalp0LrGrHRKOvTX6Ef1rU9LrTuj6cHnJE9HfcoqpFmvjW4vMWOxNHNJ9JeNMxZL0xcRwAAAKBFmttndG3PbT/pMlrtvlzQna+A9khpz/7rQzGZIOuHuHWY2W9KbJH39ZPc3LiXKpNmvjR5nZv3BZVd7/BeNT0cBrO3ZKHg99x9ZZ74U3Uh1Wn0cvBZH4atmnlRdK02ZI1XPkSbPivYDAACKYii3cLhP0oWSZptZs6Rb3P27Bfo2SvqIu18n6QxJ3zGztKL/fc/X3P2pUat8PCqvim8LcXbf9nRaOvZSdPbr8F+lw3t65596UHr1UP+xLBEFrSlz+oevyTOlSTNyHjOl8knRJU8AADBiQ7pceKqN+8uFo639FenYfulYq3S8VTrWFk9bpeNtfdu7Xy08TlllVuiaHt1PrKI6uvN9RU007WnLWpecFP0FZlllPM2er4wubRLeACA89+gnKZ6WPNU7r0Kf9Xnem82i9sGmfXfct4ZC6wr2ifulU9FNwTNTzyx3913nmeeYjsbqmc95yKSlbynw3EfPqF0uRAmqmho9Zi8buJ+71PWq9OrhPI9DOcsvR+Gsc7fUcUzqOCp1HR9+jWWVcfhKRtNE+cDzVhZd7sxMs+ezp9nhrWe+QJslokeirHfestpzHz3bKE9bIme8THtZzj5yauxTk3prK7juJBR608q3rmd5oC9ZA9TlnjN2nvl8Y+Ueo8xyZruhTDMfGO4DbJtbtxWYj6Wz37Cz5tO5b9iZN/OB6hzk+fSpP6utZznPB8fJfEj17EP5x+nXJuX/gMrpkxlfhRa998OvZ5rOWc58OGbVmvtvp19bzo76PO/s+dz/Nk2D/jeed31Zb3vB1zq3nsH+7Waer/If/yFNVaA98281fm3RV3KS9KWCPxcPv/ui7RmnnplUMTl6TFtw8tun01LnseiRCV6dR6XujuiR6oynHdHd8zPz3Z29bamuqF+6u/98Ol7f3dn/jbrnG03Om3fum3ChN+CeD42U+n2AZD/Sqb7bAdl6gvYQv+n3TKXeEJC7Lk8oyG3PFwwVj5mvxsxYme3ytmV/Ocj58pFv/z3jD/AlIZHI/2Uot90SWWPlPLd+bbkBPWef2UE9NyTK8//3XegMSJ+wnS68z3z1DOWsT59jMMTtBpwq59iV5XyJTOT/QpoxUFAfSuDr95oUXBj6v99EMuuR6Luc/aU7kdTAYTkrNBcRIQtDl0j0njUb77LPLuQ9C5Aa4A06lbWcdVYhGrj/fgqtG6i2gme88n3oFvqwyvfhNUhdffad78MxT/uAQThnzKF8sAwYdLLGyLe/fGdO8r0x9/mgyoSRQq85AORHyALyyXzTBwBgmPh/vAAAAARAyAIAAAiAkAUAABAAIQsAACAAQhYAAEAAhCwAAIAACFkAAAABELIAAAACIGQBAAAEQMgCAAAIgJAFAAAQACELAAAgAEIWAABAAIQsAACAAAhZAAAAARCyAAAAAiBkAQAABEDIAgAACICQBQAAEAAhCwAAIABCFgAAQACELAAAgAAIWQAAAAEQsgAAAAIgZAEAAAQwpJBlZuvMrNXMduRZ91kzczObXWDba8zsz/HjmpEWDAAAMBYM9UzWekmX5Daa2UJJqyU9n28jM5sp6RZJfyPpPEm3mNmMYVUKAAAwhgwpZLn7JkmH8qy6U9LnJXmBTd8q6RF3P+TuhyU9ojxhDQAAYLwZ9m+yzOwKSS3uvm2AbgskvZC13By3AQAAjGvJ4WxkZpMl3aToUuGoMLM1ktZI0qJFi0ZrWAAAgKIY7pms10haImmbme2RVC9pi5nV5fRrkbQwa7k+buvH3e9290Z3b6ytrR1mWQAAAKVhWCHL3be7+xx3X+zuixVdBnydu7+U0/VhSavNbEb8g/fVcRsAAMC4NtRbONwn6XFJy82s2cw+NEDfRjO7R5Lc/ZCkr0h6In7cFrcBAACMa+Ze6A8Di6exsdGbmpqKXQYAAMCgzGyzuzfmtnPHdwAAgAAIWQAAAAEQsgAAAAIgZAEAAARAyAIAAAiAkAUAABAAIQsAACAAQhYAAEAAhCwAAIAACFkAAAABELIAAAACIGQBAAAEQMgCAAAIgJAFAAAQACELAAAgAEIWAABAAIQsAACAAAhZAAAAARCyAAAAAiBkAQAABEDIAgAACICQBQAAEAAhCwAAIABCFgAAQACELAAAgAAIWQAAAAEQsgAAAAIgZAEAAARAyAIAAAhg0JBlZuvMrNXMdmS1fcXMnjSzrWb2KzObX2DbVNxnq5k9OJqFAwAAlLKhnMlaL+mSnLa17n62u6+S9DNJ/1hg21fdfVX8uHwEdQIAAIwpg4Ysd98k6VBO2ytZi1Mk+SjXBQAAMKYlh7uhmd0h6QOSjki6qEC3KjNrktQt6Wvu/tPh7g8AAGAsGfYP3939ZndfKGmDpI8X6HaauzdKeq+kfzWz1xQaz8zWmFmTmTW1tbUNtywAAICSMBp/XbhB0rvyrXD3lni6W9J/Sjq30CDufre7N7p7Y21t7SiUBQAAUDzDCllmtixr8QpJz+TpM8PMKuP52ZLeJOmp4ewPAABgrBn0N1lmdp+kCyXNNrNmSbdIutTMlktKS9or6SNx30ZJH3H36ySdIek7ZpZWFOa+5u6ELAAAMCGYe+n9YWBjY6M3NTUVuwwAAIBBmdnm+DfofXDHdwAAgAAIWQAAAAEQsgAAAAIgZAEAAARAyAIAAAiAkAUAABAAIQsAACAAQhYAAEAAhCwAAIAACFkAAAABELIAAAACIGQBAAAEQMgCAAAIgJAFAAAQACELAAAgAEIWAABAAIQsAACAAAhZAAAAARCyAAAAAiBkAQAABJAsdgEAAKB4urq61NzcrPb29mKXUvKqqqpUX1+v8vLyIfUnZAEAMIE1NzerpqZGixcvlpkVu5yS5e46ePCgmpubtWTJkiFtw+VCAAAmsPb2ds2aNYuANQgz06xZs07qjB8hCwCACY6ANTQn+zoRsgAAQFFVV1cXu4QgCFkAAAABELIAAEBJcHd97nOf08qVK9XQ0KD7779fkvTiiy/qggsu0KpVq7Ry5Ur97ne/UyqV0rXXXtvT98477yxy9f3x14UAAECS9E//d6ee2vfKqI555vypuuXtZw2p709+8hNt3bpV27Zt04EDB/T6179eF1xwgX74wx/qrW99q26++WalUimdOHFCW7duVUtLi3bs2CFJevnll0e17tHAmSwAAFASHnvsMV111VUqKyvT3Llz9Za3vEVPPPGEXv/61+t73/uebr31Vm3fvl01NTVaunSpdu/erRtuuEG//OUvNXXq1GKX38+QzmSZ2TpJl0lqdfeVcdtXJF0hKS2pVdK17r4vz7bXSPpSvHi7u987GoUDAIDRNdQzTqfaBRdcoE2bNumhhx7Stddeq8985jP6wAc+oG3btunhhx/WXXfdpY0bN2rdunXFLrWPoZ7JWi/pkpy2te5+truvkvQzSf+Yu5GZzZR0i6S/kXSepFvMbMbwywUAAOPV+eefr/vvv1+pVEptbW3atGmTzjvvPO3du1dz587Vhz/8YV133XXasmWLDhw4oHQ6rXe96126/fbbtWXLlmKX38+QzmS5+yYzW5zTln3Rdookz7PpWyU94u6HJMnMHlEU1u4bTrEAAGD8esc73qHHH39c55xzjsxMX//611VXV6d7771Xa9euVXl5uaqrq/X9739fLS0t+uAHP6h0Oi1J+upXv1rk6vsb0Q/fzewOSR+QdETSRXm6LJD0QtZyc9wGAAAgSTp27Jik6Gafa9eu1dq1a/usv+aaa3TNNdf0264Uz15lG9EP3939ZndfKGmDpI+PZCwzW2NmTWbW1NbWNpKhAAAAim60/rpwg6R35WlvkbQwa7k+buvH3e9290Z3b6ytrR2lsgAAAIpj2CHLzJZlLV4h6Zk83R6WtNrMZsQ/eF8dtwEAAIxrQ72Fw32SLpQ028yaFf3F4KVmtlzRLRz2SvpI3LdR0kfc/Tp3PxTf6uGJeKjbMj+CBwAAGM+G+teFV+Vp/m6Bvk2SrstaXieptG5cAQAAEBh3fAcAAAiAkAUAABAAIQsAAIwp1dXVBdft2bNHK1euPIXVFEbIAgAACGBEd3wHAADjyC9ulF7aPrpj1jVIb/vagF1uvPFGLVy4UB/72MckSbfeequSyaQeffRRHT58WF1dXbr99tt1xRVXnNSu29vbdf3116upqUnJZFLf+MY3dNFFF2nnzp364Ac/qM7OTqXTaf34xz/W/Pnz9Z73vEfNzc1KpVL68pe/rCuvvHLYT1siZAEAgCK78sor9alPfaonZG3cuFEPP/ywPvGJT2jq1Kk6cOCA3vCGN+jyyy+XmQ153G9961syM23fvl3PPPOMVq9erV27dumuu+7SJz/5SV199dXq7OxUKpXSz3/+c82fP18PPfSQJOnIkSMjfl6ELAAAEBnkjFMo5557rlpbW7Vv3z61tbVpxowZqqur06c//Wlt2rRJiURCLS0t2r9/v+rq6oY87mOPPaYbbrhBkrRixQqddtpp2rVrl974xjfqjjvuUHNzs975zndq2bJlamho0Gc/+1l94Qtf0GWXXabzzz9/xM+L32QBAICie/e7360f/ehHuv/++3XllVdqw4YNamtr0+bNm7V161bNnTtX7e3to7Kv9773vXrwwQc1adIkXXrppfrNb36j008/XVu2bFFDQ4O+9KUv6bbbbhvxfjiTBQAAiu7KK6/Uhz/8YR04cEC//e1vtXHjRs2ZM0fl5eV69NFHtXfv3pMe8/zzz9eGDRt08cUXa9euXXr++ee1fPly7d69W0uXLtUnPvEJPf/883ryySe1YsUKzZw5U+973/s0ffp03XPPPSN+ToQsAABQdGeddZaOHj2qBQsWaN68ebr66qv19re/XQ0NDWpsbNSKFStOesyPfvSjuv7669XQ0KBkMqn169ersrJSGzdu1A9+8AOVl5errq5ON910k5544gl97nOfUyKRUHl5ub797W+P+DmZu494kNHW2NjoTU1NxS4DAIBx7+mnn9YZZ5xR7DLGjHyvl5ltdvfG3L78JgsAACAALhcCAIAxZ/v27Xr/+9/fp62yslJ/+MMfilRRf4QsAAAw5jQ0NGjr1q3FLmNAXC4EAGCCK8XfZ5eik32dCFkAAExgVVVVOnjwIEFrEO6ugwcPqqqqasjbcLkQAIAJrL6+Xs3NzWprayt2KSWvqqpK9fX1Q+5PyAIAYAIrLy/XkiVLil3GuMTlQgAAgAAIWQAAAAEQsgAAAAIgZAEAAARAyAIAAAiAkAUAABAAIQsAACAAQhYAAEAAhCwAAIAACFkAAAABELIAAAACGDRkmdk6M2s1sx1ZbWvN7Bkze9LMHjCz6QW23WNm281sq5k1jWbhAAAApWwoZ7LWS7okp+0RSSvd/WxJuyR9cYDtL3L3Ve7eOLwSAQAAxp5BQ5a7b5J0KKftV+7eHS/+XlJ9gNoAAADGrNH4TdY/SPpFgXUu6VdmttnM1ozCvgAAAMaE5Eg2NrObJXVL2lCgy5vdvcXM5kh6xMyeic+M5RtrjaQ1krRo0aKRlAUAAFB0wz6TZWbXSrpM0tXu7vn6uHtLPG2V9ICk8wqN5+53u3ujuzfW1tYOtywAAICSMKyQZWaXSPq8pMvd/USBPlPMrCYzL2m1pB35+gIAAIw3Q7mFw32SHpe03MyazexDkr4pqUbRJcCtZnZX3He+mf083nSupMfMbJukP0p6yN1/GeRZAAAAlJhBf5Pl7lflaf5ugb77JF0az++WdM6IqgMAABijuOM7AABAAIQsAACAAAhZAAAAARCyAAAAAiBkAQAABEDIAgAACICQBQAAEAAhCwAAIABCFgAAQACELAAAgAAIWQAAAAEQsgAAAAIgZAEAAARAyAIAAAiAkAUAABAAIQsAACAAQhYAAEAAhCwAAIAACFkAAAABELIAAAACIGQBAAAEQMgCAAAIgJAFAAAQACELAAAgAEIWAABAAIQsAACAAAhZAAAAARCyAAAAAiBkAQAABDBoyDKzdWbWamY7strWmtkzZvakmT1gZtMLbHuJmT1rZs+Z2Y2jWTgAAEApG8qZrPWSLslpe0TSSnc/W9IuSV/M3cjMyiR9S9LbJJ0p6SozO3NE1QIAAIwRg4Ysd98k6VBO26/cvTte/L2k+jybnifpOXff7e6dkv5N0hUjrBcAAGBMGI3fZP2DpF/kaV8g6YWs5ea4LS8zW2NmTWbW1NbWNgplAQAAFM+IQpaZ3SypW9KGkRbi7ne7e6O7N9bW1o50OAAAgKJKDndDM7tW0mWS/qu7e54uLZIWZi3Xx20AAADj3rDOZJnZJZI+L+lydz9RoNsTkpaZ2RIzq5D095IeHF6ZAAAAY8tQbuFwn6THJS03s2Yz+5Ckb0qqkfSImW01s7vivvPN7OeSFP8w/uOSHpb0tKSN7r4z0PMAAAAoKZb/Sl9xNTY2elNTU7HLAAAAGJSZbXb3xtx27vgOAAAQACELAAAgAEIWAABAAIQsAACAAAhZAAAAARCyAAAAAiBkAQAABEDIAgAACICQBQAAEAAhCwAAIABCFgAAQACELAAAgAAIWQAAAAEQsgAAAAIgZAEAAARAyAIAAAiAkAUAABAAIQsAACAAQhYAAEAAhCwAAIAACFkAAAABELIAAAACIGQBAAAEQMgCAAAIgJAFAAAQACELAAAgAEIWAABAAIQsAACAAAhZAAAAAQwassxsnZm1mtmOrLZ3m9lOM0ubWeMA2+4xs+1mttXMmkar6JH60/OH9exLR9XelSp2KQAAYJxKDqHPeknflPT9rLYdkt4p6TtD2P4idz9w8qWF88WfbNczLx2VmTR/2iQtnj1Zi2dN0ZLZU7R41hQtnj1Fi2ZOVkWSE30AAGB4Bg1Z7r7JzBbntD0tSWYWpqrA/uXd5+gvbce058AJ7Tl4XLsPHNfPnnxRR17t6umTMGnBjElaOrtaZ8ybqjPnT9WZ86ZqyewpKkuMzecNAABOnaGcyRoJl/QrM3NJ33H3uwt1NLM1ktZI0qJFi4IWtXLBNK1cMK1f++HjnfrrwePacyB6/PXgCT3Xekz/7y+71ZVySVJVeUIr6qb2CV4r6mo0pTL0SwkAAMaS0Mngze7eYmZzJD1iZs+4+6Z8HeMAdrckNTY2euC68poxpUIzplTodYtm9Gnv7E7rudZjeurFV/TUvlf01ItH9PPtL+q+Pz4vSTKTlsyaopULpuns+mk6Z+F0nTV/qiZXELwAAJiogqYAd2+Jp61m9oCk8yTlDVmlrCKZiM5azZ8q/Zeozd2170h7FLr2vaKd+47oiT2H9OC2fZKiy43L5tTo7PppOnvhdJ1TP00r6qbyOy8AACaIYCHLzKZISrj70Xh+taTbQu3vVDMzLZg+SQumT9Lfnjm3p731aLuefOGInmx+Wduaj+g/nt6v/7O5WZJUUZbQGfNq1FA/TQ3xJcvT59aovIzgBQDAeGPuA1+ZM7P7JF0oabak/ZJukXRI0v+SVCvpZUlb3f2tZjZf0j3ufqmZLZX0QDxMUtIP3f2OoRTV2NjoTU0lc8eHEXF3NR9+VduaX9aTzUe07YWXtXPfKzrW0S0pOkt2xrypalgwleAFAMAYZGab3b3fLa0GDVnFMJ5CVj7ptGvPwePa3nJE25uPaHvLkf7Bq65GZy2YphV1NTp9bo2Wz63RjCkVRa4cAADkImSVuOzgtaOlN3gdbe/u6VNbU6nlc+PQVVet0+fWaNncGlXzl40AABRNoZDFp3OJSCRMS2urtbS2WlesWiAputS4/5UOPbv/qHa9dDSa7j+qH/5xr9q70j3bLpg+SUtmT9HCmZO1cOYkLZo5WQtnTNaimZM1fXL5mL2fGQAAYxkhq4SZmeqmValuWpXecnptT3s67Xrh8Ak9+1IUunbtP6a9h07o4Z0v6dDxzj5jVFcmo/A1Iwpf86dPUm1NpebUVKo2flRXJgliAACMMkLWGJRImE6bNUWnzZqi1WfV9Vl3rKNbLxw6oRcOndDz8fSFw69q94Hj+u2uNnV0p/uNV1WeiINXlWqro+A1u7pSNVVJVVclVVMZTasrk1FbZbmqq5KaXF6mBHe/BwAgL0LWOFNdmdQZ86I70udyd718okutRzvUdrRDbcfao+nRjp62v7Qd0+//elAvn+jKM3pfZlJ1RVJTKpOaXFmmKRVJTa4oi5Yr4uXK3umk8jJVJstUmUyoIplQZTKhyvKc5Xh9eVlC5WWmZFlCFWUJJctMyYRxxg0AMGYQsiYQM+u5q/3yupoB+3al0jre0a2j7d061hE/2rt1NJ4e6+jSsfZuvdLerROd3TremdKJjm6d6Eyp7WiHjnd260RHKpp2ppRKj84fWGQCVyaElWeWE4me+SiYmZKJaLk3pGVNE9YT3JJZIS6Z6A130XI0X567fdw/kTCVmaksYUrE07KEVJZIqMxMiYSUTCRUFo9VFu+3LN5Xn/Z4SpAEgPGBkIW8yssSmj65QtMnj/y2Ee6uju60Xu1MqTOVVmd3Wh3dKbV3pdURz3d0Z9rT6uhKqSvl6k5Hbd1pV1d3Wl2ptLri+e50NGZ3Kl6fSqs75T19ulNR/1e7onG6uqNpd9rVHY8dTeO+aVcqfhSbmaLAFoe0aBoHOOsNdgmLgnMU8KJtEtnz8fY986aeEFeWsy4zhlnf7a1nXvFy9r7y98/U2G9bZfrHbTl9TNGy9SzHfeLtBuwbr5PUU4epd50y/TNjSVH9/fbZ+zr02Z9lbZe1Lvt4WZ/n2ds/d7vC2/TW2PMcrG/d2c89M44pTx+COlASCFkIzsxUVV6mqvKyYpcyqHQ6Dl7ZgaxPcIuCWFcqHYUyd6UzAc1d6bSUclcqnVYqrZ7glmnrjrfvTmdPe/eVSrvSHj1SacXT3vae9Zl17nLv7Zc9n47no0f03LLH60xlxsoeJwrFPdtkjZ/Zr3v22Dn9073zKXcppwacWomccKjcEKfe9ZJ61ueGu0xw7BtW+wc8ZdrzrMsNn8oZQ/321X8cZY+TO64sq/7sAN1/H/FTLfAcB3ie/faTvx71e/59x8geP+/rkPt6KhPIh7iPrNcq3+uYOQ7KWd87n/91S1hvbdn19z0u/ceJn3He1y3f8+8dtvdYZdfaM17WfnrX9X0dk2Wmi1fMVbEQsoAsiYSpImGqEHfcD8H7hLbeAJZpc0mellzZAS5a9pzQ567okdNXPcvxunTv9lENOeP1qStrn/G67FrVZ8zeOrL3mTtmZl/Z69SnX55tep53/3GyXw/1Wd/39VBW/Zn1meeQztkmKilnzJzXqrdv39c+d9/KrrunPXvZe49DgfGVdzv1375QLWnJlc47Rs8e8jzH3P3220/WWPmfZ289Q3kd8+6v57XJv5/sY6cCtWX2j+gPu575ytuKtn9CFoBTpucynGzwzgBGrG8Izw2b3ieQFQqU7ur9gpEdALO2yYTB3HGi/fTW0Webnvm+Y2bae7fu29Zv35n5nOVSQMgCAGCcynyxiZeKWcqExDURAACAAAhZAAAAARCyAAAAAiBkAQAABEDIAgAACICQBQAAEAAhCwAAIABCFgAAQACELAAAgAAIWQAAAAGYl8r/4CeLmbVJ2ht4N7MlHQi8DwwPx6a0cXxKG8endHFsSttIjs9p7l6b21iSIetUMLMmd28sdh3oj2NT2jg+pY3jU7o4NqUtxPHhciEAAEAAhCwAAIAAJnLIurvYBaAgjk1p4/iUNo5P6eLYlLZRPz4T9jdZAAAAIU3kM1kAAADBTLiQZWaXmNmzZtC0PAQAAANoSURBVPacmd1Y7HomOjNbZ2atZrYjq22mmT1iZn+OpzOKWeNEZWYLzexRM3vKzHaa2Sfjdo5PCTCzKjP7o5lti4/PP8XtS8zsD/F73P1mVlHsWicqMyszsz+Z2c/iZY5NiTCzPWa23cy2mllT3Dbq720TKmSZWZmkb0l6m6QzJV1lZmcWt6oJb72kS3LabpT0a3dfJunX8TJOvW5Jn3X3MyW9QdLH4v9eOD6loUPSxe5+jqRVki4xszdI+mdJd7r7ayUdlvShItY40X1S0tNZyxyb0nKRu6/Kum3DqL+3TaiQJek8Sc+5+25375T0b5KuKHJNE5q7b5J0KKf5Ckn3xvP3Svrvp7QoSJLc/UV33xLPH1X0YbFAHJ+S4JFj8WJ5/HBJF0v6UdzO8SkSM6uX9N8k3RMvmzg2pW7U39smWshaIOmFrOXmuA2lZa67vxjPvyRpbjGLgWRmiyWdK+kP4viUjPhy1FZJrZIekfQXSS+7e3fchfe44vlXSZ+XlI6XZ4ljU0pc0q/MbLOZrYnbRv29LTnSAYCQ3N3NjD+BLSIzq5b0Y0mfcvdXoi/kEY5Pcbl7StIqM5su6QFJK4pcEiSZ2WWSWt19s5ldWOx6kNeb3b3FzOZIesTMnsleOVrvbRPtTFaLpIVZy/VxG0rLfjObJ0nxtLXI9UxYZlauKGBtcPefxM0cnxLj7i9LelTSGyVNN7PMF2je44rjTZIuN7M9in6WcrGk/ymOTclw95Z42qroC8p5CvDeNtFC1hOSlsV/4VEh6e8lPVjkmtDfg5KuieevkfTvRaxlwop/Q/JdSU+7+zeyVnF8SoCZ1cZnsGRmkyT9raLfzT0q6e/ibhyfInD3L7p7vbsvVvQ58xt3v1ocm5JgZlPMrCYzL2m1pB0K8N424W5GamaXKrpWXiZpnbvfUeSSJjQzu0/ShYr+7+f7Jd0i6aeSNkpaJGmvpPe4e+6P4xGYmb1Z0u8kbVfv70puUvS7LI5PkZnZ2Yp+nFum6AvzRne/zcyWKjp7MlPSnyS9z907ilfpxBZfLvwf7n4Zx6Y0xMfhgXgxKemH7n6Hmc3SKL+3TbiQBQAAcCpMtMuFAAAApwQhCwAAIABCFgAAQACELAAAgAAIWQAAAAEQsgAAAAIgZAEAAARAyAIAAAjg/wNJBvHfh93Z5AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "loss_df = pd.DataFrame(history.history)\n",
    "\n",
    "# Plot loss vs epochs\n",
    "\n",
    "loss_df.plot(figsize=(10,5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "d77qp66ChGrB",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "d77qp66ChGrB",
    "outputId": "2dd78348-3a50-4081-80ff-fe5a414f25c8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5/5 [==============================] - 0s 3ms/step - loss: 18.7532\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "18.75318717956543"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(test_data_prepared, test_label)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bhEnsQqNEmS",
   "metadata": {
    "id": "6bhEnsQqNEmS"
   },
   "source": [
    "<a name='3-7'></a>\n",
    "\n",
    "## 3.7 Saving and Loading a model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "BoN-N6yNNMqK",
   "metadata": {
    "id": "BoN-N6yNNMqK"
   },
   "source": [
    "If something went well, from training to evaluating to improving a model, you would want to save it. \n",
    "\n",
    "Here is how to save a model and how to load a saved model. The model will be saved in HDF5 `format`. When the model is saved in such format, the whole model things are saved. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "mo-NVNHRNIp0",
   "metadata": {
    "id": "mo-NVNHRNIp0"
   },
   "outputs": [],
   "source": [
    "model.save('forest_model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "htzej8JaOwiB",
   "metadata": {
    "id": "htzej8JaOwiB"
   },
   "source": [
    "And loading the model is simple too..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2ZBPS83dOvXA",
   "metadata": {
    "id": "2ZBPS83dOvXA"
   },
   "outputs": [],
   "source": [
    "loaded_model = keras.models.load_model('forest_model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "AFq80B7jhbho",
   "metadata": {
    "id": "AFq80B7jhbho"
   },
   "source": [
    "You can make predictions on a loaded model. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "JL1emoFCQ3v2",
   "metadata": {
    "id": "JL1emoFCQ3v2"
   },
   "source": [
    "<a name='3-8'></a>\n",
    "\n",
    "## 3.8 Final Notes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03YHhwcrQ-pC",
   "metadata": {
    "id": "03YHhwcrQ-pC"
   },
   "source": [
    "It has been a quite long journey, from fitting a straight line to building a neural network for a real world dataset. \n",
    "\n",
    "Ideally, for all datasets we used, neural networks would not be a suitable model. But because we are learning, it makes sense to start simple for the sake of understanding the latter. \n",
    "\n",
    "In the next lab, we will do classification with neural networks. Later we will go deep into areas that neural networks have shown potential such as computer vision and natural language processing, and that's where we will practice all possible techniques of improving the results of the neural networks."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Xwl8zemTSVx2",
   "metadata": {
    "id": "Xwl8zemTSVx2"
   },
   "source": [
    "## [BACK TO TOP](#0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6EuyW3AVSbL4",
   "metadata": {
    "id": "6EuyW3AVSbL4"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "7.3 Neural Networks for Regression with TensorFlow.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
